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outputs  and  key  Army  acquisition  documents  and  processes  are  specified. 

This  report  provides  an  overview  of  the  components  of  ETES,  describes  the 
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This  user's  guide  is  part  of  the  Early  Training  Estimation 
System  (ETES).  Development  of  the  ETES  was  sponsored  by  the 
Army  Research  Institute  (ARI)  under  contract  No.  MDA-903-80 
C-0525.  Dynamics  Research  Corporation  (DRC)  of  Wilmington, 
Massachusetts  was  the  contractor.  The  contract  monitor  for 
the  project  was  Dr.  Charles  Jorgensen.  The  conceptual 
framework  for  the  Media  Selection  Program  was  developed  by 
Dr.  Lawrence  O'Brien  and  Ms.  Donya  Boylston.  Ms.  Rose  white 
developed  the  software  for  the  Media  Selection  Program. 
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SECTION  I  -  INTRODUCTION 


1.1  OBJECTIVES 

This  guide  provides  detailed  instructions  for  using  the 
Media  Selection  Program.  This  program  is  an  automated  aid 
for  (1)  assigning  tasks  to  media,  and  (2)  determining  the 
efficiency  and  effectiveness  of  media-task  combinations. 
The  Media  Selection  Program  is  part  of  the  Early  Training 
Estimation  System  (ETES).  The  Media  Selection  Program  is 
designed  to  use  input  data  from  the  ETES  data  base  manage¬ 
ment  system  which  is  called  the  System  Description  Tech¬ 
nology  (SDT).  A  description  of  the  other  components  of  ETES 
is  provided  in  The  ETES  User's  Guide.  A  description  of  the 
SDT  is  provided  in  the  SDT  User's  Guide. 


1.2  POTENTIAL  USERS 


The  Media  Selection  Program  has  been  designed  to  be  used  by 
Army  technical  analysts  who  are  directly  concerned  with 
training  development  for  new  Army  Weapon  Systems.  The 
primary  user  organizations  are  expected  to  be:  (1)  the 
Training  Development  Directorates  in  The  Army  schools 
associated  with  development  of  new  systems,  (2)  Program 
Manager's  Office  for  new  systems,  particularly  those 
individuals  concerned  with  training  development,  (3)  other 
Army  organizations  concerned  with  training  development  such 
as  the  TRADOC  Systems  Analysis  Activity  (TRASANA)  and  PM 
Trade,  and  (4)  contractors  who  must  develop  training 
requirements  for  new  systems.  It  is  assumed  that  users  are 
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familiar  with  the  basic  Army  training  development 
terminology  and  processes. 


1.3  CONCEPTUAL  OVERVIEW 

The  Media  Selection  Program  is  an  automated  tool  for  (1) 
assigning  tasks  to  media  and  (2)  calculating  the  efficiency 
and  effectiveness  of  various  task-media  combinations.  The 
media  Selection  Program  will  allow  you  to  assign  tasks  to 
media  in  a  manner  that  maximizes  overall  efficier 
maximizes  overall  effectiveness,  or  minimizes  ove  11 

cost.l  in  addition,  it  would  allow  you  to  assign  task  to 
media  in  a  manner  that  optimizes  various  combination 
these  variables,  including  an  overall  "utility"  measure 
which  combines  either  cost  and  efficiency  or  cost  and 
effectiveness . 

Efficiency,  in  the  Media  Selection  Program,  is  determined  by 
comparing  the  stimulus,  response,  and  feedback  character¬ 
istic  of  the  tasks  to  the  stimulus,  response,  and  feedback 
characteristics  of  potential  media.  More  specifically,  a 
score  is  calculated  which  describes  the  match  between  media 
and  tasks  on  these  characteristics.  Efficiency  for  each 
task-media  combination  is  calculated  by  dividing  this  score 
by  the  maximum  match  that  may  be  achieved  for  the  task. 


1  The  program  uses  a  relative  cost  and  not  actual  cost  to 
measure  the  potential  cost  requirements  of  media.  The 
relative  cost  values  used  in  the  program  are  listed  in 
Appendix  B. 


Total  efficiency  for  a  set  of  tasks  is  the  sum  of  the 

2 

efficiency  score  for  individual  tasks. 

Effectiveness  is  calculated  by  weighting  the  efficiency  of 
each  task  by  a  task  criticality  score.  The  task  criticality 
score  is  a  user-defined  weighted  combination  of  the  eight 
task  factors  typically  used  in  selecting  tasks  for  training. 
These  eight  factors  are  task  frequency,  per  cent  members 
performing,  per  cent  time  performing,  task  delay  tolerance, 
consequences  of  inadequate  performance,  task  learning 
difficulty,  probability  of  deficient  performance,  and  time 
between  entry  and  performance. 

A  matrix  of  relative  cost  values  is  stored  in  the  program 
for  each  major  media  category.  (See  Appendix  B  for  a 
listing  of  these  cost  values).  In  addition,  a  built-in  set 
of  algorithms  is  used  to  produce  the  utility  measure  which 
combines  cost  with  either  effectiveness  or  efficiency.  (See 
Appendix  A  for  a  description  of  these  algorithms). 

1.4  PROCEDURAL  OVERVIEW 

The  Media  Selection  Program  is  an  interactive  menu-driven 
system.  This  means  that  you  do  not  have  to  know  or  use  a 
computer  language  to  run  the  program.  Instead,  you  can  run 
through  the  program  by  selecting  options  from  a  series  of 
menus.  The  program  will  tell  you  what  information  to  enter 


'  This  concept  of  media  efficiency  was  originally 
developed  by  Jorgensen  (1978)  and  was  incorporated  into 
Training  Efficiency  Estimation  Model  (TEEM)  described  in 


and  when  to  enter  this  information.  It  will  tell  you  when 
you  have  data  missing  and  will  indicate  when  you  have 
entered  data  that  is  of  the  wrong  type  or  when  you  have 

entered  data  that  is  out  of  the  acceptable  range  for  the 

variable  you  are  trying  to  enter. 

An  overview  of  the  procedures  for  using  the  Media  Selection 
Program  is  provided  in  Figure  1-1.  To  begin  the  procedures, 
you  must  rate  each  task  on  (1)  the  psychological  variables 
to  be  used  to  assess  the  match  between  tasks  and  media,  and 
(2)  the  variables  to  be  used  to  assess  task  criticality  (the 
latter  is  only  necessary  if  effectiveness  is  being 

calculated).  These  scores  must  be  then  entered  into  the 
ETES  data  base  management  system,  the  System  Description 
Technology.  When  this  is  completed  you  must  enter  the 
Applications  Program  mode  in  the  System  Description 

Technology,  and  select  the  Media  Selection  Program.  Once  in 
the  Media  Selection  Program,  you  must  then  select  criteria 
to  be  used  in  making  Media  assignments.  Seven  options  for 
selecting  criteria  are  provided:  (1)  efficiency,  (2) 

effectiveness  (3)  relative  cost,  (4)  cost  and  efficiency, 
(5)  cost  and  effectiveness,  (6)  a  utility  measure,  combining 
efficiency  and  cost,  and  (7)  a  utility  measure  combining 
effectiveness  and  cost. 

Once  the  criteria  have  been  selected,  you  must  select  the 
tasks  to  be  included  in  the  analysis  (only  tasks  already  in 
the  SDT  may  be  selected).  Typically,  the  tasks  for  single 
course  module  will  be  selected  for  each  analysis. 

With  the  analysis  criteria  identified,  the  psychological 
variables  to  be  used  in  calculating  the  match  between  tasks 
and  media  must  be  selected.  You  may  select  from  twelve 


variables  assessing  stimulus  charac ter ist ics ,  six  variables 
assessing  response  characteristics  and  four  variables 
assessing  feedback  characteristics. 3 

If  you  have  selected  a  set  of  criteria  involving  effective¬ 
ness  (effectiveness,  cost  and  effectiveness,  or  utility  with 
effectiveness),  the  weights  for  the  component  criticality 
variables  must  be  entered  so  that  a  composite  task 
criticality  score  can  be  computed.  Eight  variables  may  be 
used  in  the  calculation  of  task  criticality:  (task 
frequency,  per  cent  member  performing,  per  cent  time 
performing,  task  delay  tolerance,  consequences  of  inadequate 
performance,  task  learning  difficulty,  probability  of 
deficient  performance  and  time  between  entry  and 
performance)  . 

At  this  point,  the  program  has  all  of  the  information  needed 
to  calculate  efficiency  and/or  effectiveness.  It  will  use 
this  information  to  calculate  the  relevant  measure  and  it 
will  display  the  results. 

If  you  have  selected  one  of  the  two  utility  measures 
(combining  either  cost  and  efficiency  or  cost  and  effective¬ 
ness)  you  will  then  be  required  to  enter  the  weights  to  be 
used  in  the  computation  of  utility.  Following  this  you  must 
select  the  objectives  and  constraints  to  be  used  in  assign¬ 
ing  tasks  to  media.  A  listing  of  the  possible  combinations 
of  objectives  and  constraints  is  displayed  in  Table  1-1. 
(More  information  on  strategies  for  selecting  objectives  and 


J  These  psychological  variables  were  taken  directly  from 
the  TEEM  Model  (Jorgensen,  Kubula,  and  Atlas;  1981). 
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constraints  is  provided  in  Section  2.5).  Once  this 
information  has  been  put  into  the  computer,  the  program  will 
optimally  assign  the  tasks  to  training  media.  For  instance, 
if  you  selected  "maximize  effectiveness"  as  your  objective 
and  "minimize  cost"  as  your  constraint,  the  program  would 
determine  the  assignment  of  tasks  to  media  which  gives  the 
highest  overall  score  on  effectiveness  and  still  remains 
under  a  user-specified  level  of  overall  cost.  Once  the 
initial  assignments  have  been  examined,  you  may  wish  to 
examine  the  effects  of  several  alternatives  including 
changes  in  (1)  objective,  (2)  constraint,  (3)  criteria,  (4) 
task  criticality  variables,  (5)  psychological  variables,  (6) 
task  criticality  variable  weights,  and/or  (7)  utility 
weights . 

After  you  have  explored  these  alternatives,  the  task-media 
assignments  from  the  Media  Selection  Program  should  be 
compared  against  practical  factors,  such  as  resource  avail¬ 
ability,  and  schedule  and  budget  consti*  'nts  and  a  final  set 
of  assignments  should  be  constructed  and  entered  into  the 
SDT. 


1.5  PRODUCTS  OF  MEDIA  SELECTION  PROGRAM 

The  Media  Selection  Program  will  produce  two  major  products: 
(1)  a  listing  of  the  efficiency,  effectiveness  or  utility 
scores  for  each  media-task  combination  and  (2)  a  listing  of 
the  optimal  media-task  assignments.  Examples  of  both  of 
these  products  are  provided  in  Tables  1-2,  and  1-3 
respectively . 


1.6  GUIDELINES  FOR  USING  THIS  GUIDE 


The  remainder  of  this  User's  Guide  is  divided  into  four 
sections.  Section  2  describes  the  basic  operational 

procedures  for  using  the  Media  Selection  program  and  the 
hardware  and  software  which  are  required  to  run  the  program, 
and  procedures  for  developing  the  input  data  needed  to  run 
the  program.  Section  3  provides  a  detailed  description  of 
the  menus  and  frames  in  the  Media  Selection  Program. 
Section  4  contains  an  example  application  of  the  Media 
Selection  Program.  This  example  is  designed  to  provide 
first-time  users  with  some  hands-on  experience  in  using  the 
program.  No  experience  is  required  to  run  this  application. 
Section  5  contains  guidance  on  evaluating  and  using  the 
results  of  the  Media  Selection  Program.  Appendix  A  provides 
a  technical  description  of  the  Media  Selection  Program 
assignment  algorithms  and  procedures.  Appendix  B  provides  a 
detailed  description  of  the  psychological  variables, 
criticality  variables,  media  categories  and  relative  cost 
values  which  are  used  in  the  Media  Selection  Program. 


SECTION  2 


BASIC  OPERATIONAL  PROCEDURES 


This  section  describes  basic  operational  procedures  for 
using  the  Media  Selection  Program.  In  addition,  it 
describes  the  hardware/software  required  to  use  the  program 
and  the  input  data  which  must  be  developed  before  the 
program  can  be  run.  This  section  is  divided  into  five 
subsections.  Section  3.1  describes  the  hardware  required  to 
use  the  program.  Section  3.2  describes  the  software 
required  to  use  the  program.  Section  3.3  describes 
procedures  for  using  the  program  menus.  Section  3.4 
describes  procedures  for  loading  the  Media  Selection 
Program.  Section  3.5  describes  the  input  data  which  must  be 
developed  to  run  the  program. 


2.1  HARDWARE 

To  use  the  Media  Selection  Program,  you  must  have  an  Apple 
III  computer  with: 

o  128K  Bytes  of  RAM  memory. 

o  A  video  monitor.  The  monitor  can  be  black  and 
white  or  color.  However,  the  program  does  not 
produce  color  images. 

o  A  5  megabyte  Profile  hard  disk. 

o  An  additional  floppy  disk  drive  (that  is,  over  and 
above  the  disk  drive  built  into  your  Apple  III). 
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This  drive  is  needed  to  make  a  local  floppy  disk 
copy  of  an  SDT  data  base  and  to  load  the  System 
Description  Technology  (SDT)  software  which 
contains  the  SDT  data  base  management  system  and 
the  Media  Selection  Program. 

o  A  printer.  A  printer  is  needed  to  get  hard  copy 
output . 

If  you  are  using  the  Apple  for  the  first  time,  be  sure  to 
follow  the  APPLE  III  Owner's  Guide  for  instructions  on  how 
to  connect  the  computer,  the  monitor,  and  an  extra  floppy 
disk  drive.  Also,  follow  the  manufacturer's  instructions  on 
how  to  connect  your  PROFILE  hard  disk  to  your  system. 

Procedures  for  insuring  that  your  printer  and  other  peri¬ 
pherals  are  properly  linked  to  SDT  software  are  described  in 
the  SDT  User  Guide. 


2.2  SOFTWARE  REQUIREMENTS 


To  use  the  Media  Selection  Program,  you  will  need  the 
following  software:* 

o  SDT  Program  Diskettes.  The  program  diskettes 
contain  software  for  both  the  SDT  and  the  Media 
Selection  Program. 


1  For  information  on  obtaining  SDT  software,  contact  Dr. 
Lawrence  O'Brien,  Dynamics  Research  Corporation,  7  Lopez 
Road,  Wilmington,  MA  01887. 


o  SDT  Backup  Diskette  (Optional)  -  These  backup 
diskettes  are  used  to  make  a  local  copy  of  an  SDT 
data  base. 


2.2.1  General  Instructions  for  Handling  Diskettes 

To  insert  a  diskette,  open  the  disk  drive  and  slip  the 
diskette  into  the  slot  with  the  label  facing  upward  as  shown 
in  Figure  2-2.  The  edge  of  the  diskette  with  the  oval 
cutout  should  enter  the  drive  first;  the  edge  with  the  label 
should  enter  face  up  and  last.  Gently  push  the  diskette 
into  the  drive;  do  not  bend  it.  Close  the  drive  door 
firmly . 

To  remove  the  diskette,  open  the  door  and  pull  the  diskette 
straight  out  of  the  slot.  If  you  leave  the  diskette  in  a 
drive  for  long  periods  of  time  without  use,  it  is  a  good 
idea  to  open  the  door  so  the  read/write  head  does  not  rest 
on  the  diskette.  NEVER  REMOVE  A  DISKETTE  WHILE  THE  RED 
LIGHT  UNDER  THE  DOOR  IS  ON.  This  can  permanently  damage  the 
diskette  and  is  almost  certain  to  destroy  the  information  on 
it.  You  may  be  able  to  reuse  such  a  diskette,  but  you  will 
not  be  able  to  recover  the  lost  data. 


2.2.2  Loading  Program  Software 

The  software  which  contains  the  SDT  and  Media  Selection 
programs  should  be  stored  on  your  PROFILE  hard  disk. 

To  load  the  SDT  program  into  your  hard  disk  use  the 
following  procedures; 

i 

i 

I 

l 

l 
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o  Load  the  SDT  Boot  Disk  into  the  built-in  drive. 
Turn  the  computer  off  and  then  on  again  using  the 
switch  at  back  left  of  the  Apple. 

o  Follow  the  instructions  which  appear  on  the 
screen,  and  load  the  SDT  Program  Diskettes  into 
the  additional  floppy  disk  drive. 

Once  the  SDT  Program  Diskettes  have  been  loaded  into  your 
PROFILE  hard  disk,  you  will  not  have  to  enter  the  SDT 
Program  Diskettes  again. 

2.3  PROGRAM  MENUS 

Unlike  many  computer  programs,  you  do  not  have  to  know  a 
computer  language  to  use  the  Media  Selection  Program. 
Instead,  you  can  make  commands  and  move  through  the  program 
by  selecting  options  from  a  series  of  menus.  An  example 
menu  is  listed  in  Figure  2-3. 

To  select  a  menu  option,  you  must  highlight  it  with  the 
cursor  and  press  the  RETURN  key.  The  cursor  is  a 
highlighted  area  on  the  screen,  as  shown  in  Figure  2-3. 

You  can  move  the  cursor  by  hitting  the  up  or  down  arrow 
keys.  Moving  the  cursor  to  a  menu  option  causes  no  action 
to  take  place.  To  select  a  menu  option,  you  must  move  the 
cursor  to  the  option  and  press  the  RETURN  key. 

Some  menus  in  the  program  may  have  so  many  items  that  not 
all  of  these  items  will  be  able  to  be  shown  at  the  screen  at 
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Return  to  SDT 
Continue  with  Program 


Figure  2-3.  Typical  Menu 


any  one  time  (such  menus  will  have  the  phrase  MORE  at  the 
bottom).  To  see  these  additional  items,  you  must  scroll  the 
menu  up  or  down.  To  scroll  down,  move  to  the  bottom  of  the 
menu  by  hitting  the  down  arrow  key.  Continue  to  hit  this 
key  and  the  menu  will  move  up  showing  you  the  additional 
items  in  the  menu.  You  can  then  scroll  back  up  by  hitting 
the  up  arrow  key. 


2.4  LOADING  THE  MEDIA  SELECTION  PROGRAM 


To  run  the  Media  Selection  Program,  you  must  do  three 
things.  First,  you  must  load  the  SDT  Boot  Diskette  into 
your  built-in  drive.  Second  you  must  load  the  five  SDT 
Program  Diskettes  onto  your  PROFILE  hard  disk.  Procedures 
for  accomplishing  these  first  two  steps  were  described  in 
Section  2.2.  When  you  have  successfully  accomplished  these 
two  steps  you  will  see  the  first  menu  of  the  SDT  data  base 
management  system  on  your  screen. 

In  the  third  step,  you  must  enter  the  SDT  program  and  move 
your  way  through  the  program  to  the  Applications  Program 
mode  where  you  can  select  the  Media  Selection  program  and 
load  this  program  into  "active  memory."  To  do  this  you  must 
run  through  four  SDT  menus.  On  the  pages  which  follow  these 
four  menus  are  listed  and  the  specific  options  you  must 
select  on  each  menu  are  described.  When  you  are  finished 
with  these  four  menus,  the  first  menu  or  frame  of  the  Media 
Selection  Program  will  appear  on  the  screen  (This  frame  will 
say  "Welcome  to  Media  Selection  Program"  at  the  top). 
Details  for  using  the  menus  and  frames  in  this  program  are 
provided  in  Section  3. 


Action  Is 


Examine  SDT  Introduction  Frame 


The  SDT  introductory  frame  (see  Figure  2-4)  will  appear  on 
the  screen  after  you  have  loaded  the  SDT  Program  Diskettes 
and  SDT  Boot  Diskette  (see  Section  2.2).  The  frame  will 
describe  instructions  for  using  SDT  menus.  Read  the 
instructions  and  use  what  you  have  learned  to  move  to  the 
next  frame.  More  specifically,  move  the  cursor  up  and  down 
until  you  get  to  the  option  Skip  to  Start  of  Program,  then 
hit  RETURN. 

Action  2:  Select  System 

A  menu  will  appear  on  the  screen  asking  you  to  select  the 
system  to  be  examined  (See  Figure  2-5  for  an  example).  Move 
the  cursor  to  the  system  you  want  to  work  with  and  hit 
RETURN .  (See  the  SDT  User's  Guide  for  instructions  on 
setting  up  an  SDT  data  base  for  your  weapon  system). 

Action  3:  Examine  System  Description 

A  description  of  the  data  base  you  selected  will  appear  on 
the  screen  (see  example  in  Figure  2-6).  Read  the 
description,  then  move  the  cursor  to  Continue  and  hit 
Return. 


Action  4:  Select  SDT  Mode  of  Operation 

A  menu  will  appear  on  the  screen  asking  you  to  select  one  of 
the  seven  SDT  modes  of  operation  (see  Figure  2-7).  Move  the 
cursor  to  Applications  and  hit  Return. 
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( SDT) 


Whenever  the  right  arrow  appears,  press  the  up  or  down 
arrow  keys  on  the  keyboard  or  use  the  joystick  until  the 
desired  menu  action  is  highlighted.  Then  press  the  RETURN 
key  on  the  keyboard  or  push  the  joystick  button. 

For  a  further  explanation  at  any  time,  hold  down  the  escape 
key  (upper  left  hand  corner)  or  toggle  the  auxiliary  switch 
on  the  joystick.  The  explanation  will  remain  on  the  screen 
for  as  long  as  the  key  or  button  is  held  down. 

More  instructions 
Skip  to  start  of  program 
Wait  until  later  time 


Figure  2-4.  SDT  Introduction  (SDT-1) . 
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SYSTEM  SELECTION 


Select  system  for  this  session. 


-  SDT  Example 

-  ARI  Example 


Figure  2-5.  System  Selection 


System  Selected: 

SDT  EXAMPLE 
DESCRIPTION  GOES  HERE. 
UP  TO  10  LINES 


-  ABORT  SDT  EXAMPLE - 
CONTINUE 


Figure  2-6.  System  Description  (SDT-3) 


SDT  MODE  SELECTION 


SELECT  SDT  OPERATION  MODE: 


-EXIT  SDT- 
-AUDIT/UPDATL- 
-  INPUT  DATA- 
-CORRECT  DATA- 
-OUTPUT  DATA- 
-SAVE/RESTORE- 
-APPLICATIONS- 


(Terminate  the  Program) 

(Examine  Audit  Trail/Update  on  Honeywell  at  DRC) 
(Add  Tabular  Data  into  the  SDT  data  files) 

(Change  Specific  Attribute  for  a  Particular  Entity) 
(Display  or  Print  Existing  Data) 

(Copy  this  system  to  or  from  diskette) 

(Execute  the  Applications  Programs) 


Action  5: 


Select  Media  Selection  Program 


A  menu  will  appear  on  the  screen  as  asking  you  to  select  a 
SDT  applications  program  (see  Figure  2-8).  Move  the  cursor 
to  Media  Selection  Program  and  hit  Return. 


2.5  PREPARING  INPUT  DATA 

Figure  2-9  outlines  the  steps  which  must  be  taken  to  prepare 
input  data  for  the  Media  Selection  Program.  More  details  on 
each  of  these  steps  follow. 

Select  Tasks:  You  must  select  the  tasks  to  be  used  in  the 
analysis.  (You  can  only  select  tasks  which  are  already  in 
the  SDT).  The  best  strategy  is  to  select  all  of  the  tasks 
associated  with  a  particular  course  module.  This 
information  may  be  obtained  from  the  standard  SDT  output 
report  labeled  "Quasi  Program  of  Instruction- Part  2".  This 
report  is  listed  under  the  SDT  entity  Courses.  (See  the  SDT 
User's  Guide  for  instructions  on  obtaining  this  report).  A 
maximum  of  fifty  tasks  may  be  examined  in  any  one 
application  of  the  Media  Selection  Program. 

Select  Criteria:  You  may  select  from  one  of  seven  criteria 
for  assessing  the  "goodness"  of  a  Media  task  assignment  (1) 
efficiency,  (2)  effectiveness,  (3)  cost  and  effectiveness, 
(4)  cost  and  efficiency,  (5)  relative  cost,  (6)  a  utility 
measure  combining  cost  and  efficiency  and  (7)  a  utility 
measure  combining  cost  and  effectiveness. 

Table  2-1  summarizes  the  advantages  and  disadvantages  of 
each  criteria.  More  details  on  the  advantages  and 
disadvantages  of  the  different  criteria  are  contained  in  the 
remaining  portion  of  this  discussion  for  inputting  data. 
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SDT  APPLICATIONS  PROGRAM 


Resource  and  Cost  Estimation  (Interface  with  VISICALC  Program) 
Technique 


Figure  2-8.  Select  Applications  Program. 
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Select  Stimulus,  Response,  and  Feedback  Variables:  if  you 
have  selected  an  option  involving  efficiency  or  effective¬ 
ness  (this  will  include  all  but  the  cost  only  option),  you 
must  determine  what  stimulus,  response  and  feedback 
variables  will  be  used  to  calculate  the  match  between  Media 
and  tasks.  A  listing  of  the  stimulus  response,  and  feedback 
variables  in  the  SDT  and  the  Media  Selection  Program  is 
provided  in  Table  2-2.1  Definitions  and  descriptions  of 
each  of  these  variables  is  provided  in  Appendix  B.  Unless 
you  have  extensive  experience  in  Media  selection,  it  is 
recommended  that  you  use  all  of  these  variables.  The 
criteria  you  select  should  be  documented  in  the  Media 
Selection  Program  planning  worksheet  listed  in  Table  2-3. 

Select  Criticality  Variables:  Typically,  there  are  eight 

potential  variables  which  are  used  to  determine  task 
criticality  in  the  Army:  (1)  task  frequency,  (2)  per  cent 
members  performing,  (3)  per  cent  time  performing,  (4)  task 
delay  tolerance,  (5)  consequences  of  inadequate  performance, 
(6)  task  frequency,  (7)  probability  of  deficient 
performance,  and  (8)  time  between  entry  and  performance. 
Definitions  for  each  of  these  variables  is  provided  in 
Appendix  B.  During  the  earliest  portion  of  the  acquisition 
process,  it  is  recommended  that  three  factors--task  learning 
difficulty,  consequences  of  inadequate  performance,  and  task 
frequency — be  used  to  assess  criticality.  In  later  phrases 
of  the  acquisition  process,  it  is  recommended  that  all  eight 
variables  be  used  to  assess  criticality.  More  detailed 
descriptions  of  strategies  for  assessing  task  criticality 


1  These  categories  were  taken  directly  from  the  TEEM 
model  (Jorgensen,  Kubala  and  Atlas;  1981). 


Table  2-2.  Psychological  Variables 

STIMULUS  VARIABLES 

MEDIUM 

VISUAL  FORM 

VISUAL  MOVEMENT 

VISUAL  SPECTRUM 

VISUAL  SCALE 

VISUAL  CONTRAST 

AUDIO  SOURCES 

AUDIO  STIMULI  INTENSITY 

STIMULI  PRESENTATION 

STIMULI  PRESENTATION  RATE 

NUMBER  OF  CHANNELS 

STIMULI  DISTRIBUTION 

RESPONSE  VARIABLES 
RESPONSE  MODE 
INTENSITY  OF  RESPONSE 
RESPONSE  IMPLEMENTATION 
REQUIRED  RESPONSE  RATE 
#  RESPONSE  CHANNELS 
RESPONSE  DISTRIBUTION 

FEEDBACK  VARIABLES 
FEEDBACK  MEDIUM 
FEEDBACK  SOURCE 
FEEDBACK  TYPE 
FEEDBACK  DISTRIBUTION 
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Table  2-3.  Media  Selection  Planning  Worksheet  (MSEL) . 


Date  _  Analysis  _ 

Criteria  (Pick  one)  Utility  Weights 

_ Efficiency  _ 

_ Effectiveness  _ 

_ Cost  _ 

_ Cost  and  Efficiency  _ 

_ Cost  and  Effectiveness  _ 

_ Utility  with  Efficiency  _ 

_ Utility  with  Effectiveness 

Criticality  Variables  Criticality  Weights 

_ Task  Frequency  _ 

_ Per  Cent  Members  Performing  _ 

_ Per  Cent  Time  Performing  _ 

_ Task  Delay  Tolerance  _ 

_ Task  Learning  Difficulty  _ 

_ Probability  of  Deficiency  _ 

Performance 

_ Time  Between  Entry  and  Performance  _ 

_ Consequences  of  Inadequate  _ 

Performance 


Objective  Constraint 

Maximize  Efficiency  _  _ 

Maximize  Effectiveness  _  _ 

Minimize  Cost  _  _ 

Maximize  Utility  _  _ 
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Table  2-3.  (cont'd) 


PSYCHOLOGICAL  VARIABLES 
STIMULUS  VARIABLES 

MEDIUM 

VISUAL  FORM 

VISUAL  MOVEMENT 

VISUAL  SPECTRUM 

VISUAL  SCALE 

VISUAL  CONTRAST 

AUDIO  CONTRAST 

AUDIO  SOURCES 

AUDIO  STIMULI  INTENSITY 

STIMULI  PRESENTATION 

STIMULI  PRESENTATION  RATE 

NUMBER  OF  CHANNELS 

STIMULI  DISTRIBUTION 

RESPONSE  VARIABLES 
RESPONSE  MODE 
INTENSITY  OF  RESPONSE 
RESPONSE  IMPLEMENTATION 
REQUIRED  RESPONSE  RATE 
#  RESPONSE  CHANNELS 
RESPONSE  DISTRIBUTION 

FEEDBACK  VARIABLES 
FEEDBACK  MEDIUM 
FEEDBACK  SOURCE 
FEEDBACK  TYPE 
FEEDBACK  DISTRIBUTION 


Table  2-3.  Media  Categories,  (continued) 


CAT 

# 

GROUP 

# 

1 

1.1 

2 

1.2 

3 

1.3 

4 

1.4 

5 

1.5 

6 

2.1 

7 

2.2 

8 

3.1 

9 

3.2 

10 

3.3 

11 

3.4 

12 

4.1 

13 

4.2 

14 

4.3 

IS 

4.4 

16 

4.5 

17 

4.6 

18 

4.7 

19 

4.8 

20 

4.9 

21 

4.10 

22 

5.1 

23 

5.2 

24 

5.3 

25 

5.4 

26 

5.5 

27 

5.6 

28 

5.7 

29 

5.8 

30 

8.1 

MEDIA  CATEGORY 

PRINT  INSTRUCTIONS  W/O  FEEDBACK 

PRINT  INSTRUCTIONS  WITH  ORAL  FEEDBACK 

PRINT  INSTRUCTION  -  WITH  WRITTEN  FEEDBACK 

PRINT  -  REFERENCE  MATERIAL 

IMAGE  PROJECTION  -  STILL 

PASSIVE  AUOIO 

ACTIVE  AUDIO 

AUDIOVISUAL  -  STILL  WITH  FEEDBACK 
AUDIOVISUAL  -  STILL  WITHOUT  FEEDBACK 
AUDIOVISUAL  -  MOTION  WITH  FEEDBACK 
AUDIOVISUAL  -  MOTION  WITHOUT  FEEDBACK 
STATIC  DISPLAY  WITHOUT  FEEDBACK 
DYNAMIC  DISPLAY  WITHOUT  FEEDBACK 
PHYSIOLOGICAL  TRAINER  -  INTERNAL 
PHYSIOLOGICAL  TRAINER  -  AUDIO 
PHYSIOLOGICAL  TRAINER  -  VISUAL 
SYMBOLIC  SIMULATION  WITH  FEEDBACK 
TRAINER 
SIMULATOR 

OPERATIONAL  EQUIPMENT 
OPERATIONAL  EQUIPMENT  WITH  FEEDBACK 
VISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 
VISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 
VISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 
VISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 
INSTRUCTOR 
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are  provided  in  The  ETES  User's  Guide.  The  criteria  you 
select  should  be  documented  in  the  Media  Selection  Program 
planning  worksheet  (see  Table  2-3). 

Enter  Data  Into  SDT:  Once  the  stimulus,  response,  and 
feedback  variables  and  task  criticality  variables  have  been 
selected,  each  task  must  be  scored  on  each  of  these 
variables  and  the  resulting  data  must  be  entered  into  the 
SDT.  Table  2-4  displays  a  worksheet  which  can  be  used  to 
assist  users  #in  inputting  this  data  into  the  SDT. 

To  enter  data  into  the  SDT,  enter  the  INPUT  mode  of  the  SDT 
and  select  the  Task  entity.  The  psychological  variables  and 
the  criticality  variables  are  stored  under  the  Task  entity. 
Additional  assistance  can  be  provided  by  successively 
selecting  four  built-in  input/output  formats  for  data 
entry:  (1)  Training  Emphasis  Ratings  -  this  format  contains 
the  seven  criticality  variables,  (2)  Stimuli  (Parts  1  and  2) 

-  this  format  contains  the  stimulus  variables,  (3)  Responses 

-  this  format  contains  the  response  variables,  (4) Feedback  - 
this  format  contains  the  feedback  variables.  By  selecting 
these  formats,  you  can  avoid  the  process  of  selecting  the 
variables  to  be  entered  into  the  SDT.  More  details  on 
inputting  data  into  the  SDT  are  provided  in  Section  7  of  the 
SDT  User's  Guide. 

Select  Media  Categories:  After  entering  the  task  data  into 
the  SDT,  you  must  decide  what  potential  media  categories 
will  be  considered  during  the  assignment  process.  Table  2-5 
lists  the  media  categories  which  are  available  in  the  Media 
Selection  Program.  More  detailed  descriptions  of  each  of 
these  categories  is  provided  in  Appendix  B.  To  select  a  set 
of  media  categories,  start  with  the  full  set  of  categories 
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Table  2-5.  Media  Categories. 


GROUP 

#  MEDIA  CATEGORY 

1.1  PRINT  INSTRUCTIONS  W/O  FEEDBACK 

1.2  PRINT  INSTRUCTIONS  WITH  ORAL  FEEDBACK 

1.3  PRINT  INSTRUCTION  -  WITH  WRITTEN  FEEDBACK 

1.4  PRINT  -  REFERENCE  MATERIAL 

1.5  IMAGE  PROJECTION  -  STILL 

2.1  PASSIVE  AUOIO 

2.2  ACTIVE  AUDIO 

3.1  AUDIOVISUAL  -  STILL  WITH  FEEDBACK 

3.2  AUDIOVISUAL  -  STILL  WITHOUT  FEEDBACK 

3.3  AUDIOVISUAL  -  MOTION  WITH  FEEDBACK 

3.4  AUDIOVISUAL  -  MOTION  WITHOUT  FEEDBACK 

4.1  STATIC  DISPLAY  WITHOUT  FEEDBACK 

4.2  DYNAMIC  DISPLAY  WITHOUT  FEEDBACK 

4.3  PHYSIOLOGICAL  TRAINER  -  INTERNAL 

4.4  PHYSIOLOGICAL  TRAINER  -  AUDIO 

4.5  PHYSIOLOGICAL  TRAINER  -  VISUAL 

4.6  SYMBOLIC  SIMULATION  WITH  FEEDBACK 

4.7  TRAINER 

4.8  SIMULATOR 

4.9  OPERATIONAL  EQUIPMENT 

4.10  OPERATIONAL  EQUIPMENT  WITH  FEEDBACK 

5.1  VISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 

5.2  VISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 

5.3  VISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 

5.4  VISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 

5.5  AUDIOVISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 

5.6  AUDIOVISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 

5.7  AUDIOVISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 

5.8  AUDIOVISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 

6.1  INSTRUCTOR 


listed  in  Table  2-4  and  eliminate  any  categories  which  (1) 
are  not  available  at  the  setting  where  the  course  associated 
with  tasks  is  likely  to  be  taught,  and  (2)  are  clearly 
inappropriate  for  the  particular  subject  area  covered  by  the 
course  (the  latter  criteria  should  be  used  with  extreme 
caution  since  you  may  unnecessarily  eliminate  potential 
media) . 

Once  the  media  categories  have  been  selected,  they  should  be 
documented  in  the  Media  Selection  Program  Planning  Worksheet 
listed  in  Table  2-5. 


Select  Criticality  and  Utility  Weights: 


You  must  decide 


what  weights  will  be  used  in  aggregating:  (1)  the 

criticality  variables  and  (2)  the  components  of  utility 
(that  is,  cost  and  efficiency,  or  cost  and  effectiveness). 
It  is  recommended  that  you  use  equa  1  weights  for  both  of 
these  areas. 

If  you  (1)  suspect  the  validity  of  some  of  the  variables  or 
(2)  feel  that  some  of  the  variables  are  less  important  to 
your  particular  analysis,  you  may  want  to  assign  these 
variables  smaller  weights.  However,  if  you  decide  to  do 
this  you  must  compensate  by  increasing  the  weight  of  some  of 
the  other  variables  since  the  sum  of  any  set  of  weights  must 
be  100.  The  worksheet  listed  in  Table  2-4  should  be  used  to 
document  the  criticality  weights  you  develop. 


Select  Objectives  and  Constraints:  Table  2-6  displays  the 
objectives  and  instruments  that  you  can  use  to  assign  tasks 
to  media,  the  constraints  that  can  be  used  to  qualify  each 
objective  and  the  type  of  assignments  that  will  be  produced 
by  each  objective-constraint  combination.  Examine  this 
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to  a  user-specified  level  of  effectiveness. 

utility"  An  assignment  of  all  tasks  to  media  is  generated  such  that  total 

cost  is  minimized  and  total  "utility"  is  greater  than  or  equal 
to  a  user-specified  level  of  "utility." 


table,  determine  the  types  of  assignments  you  would  like  to 
produce,  and  select  the  appropriate  combination  of  objective 
and  constraint.  The  combinations  that  are  most  likely  to 
produce  meaningful  and  easy-to-interpret  results  are  starred 
in  T&ble  2-6.  It  is  recommended  that  first-time  users 
select  one  of  the  following  combinations; 

OBJECTIVE  CONSTRAINT 

Maximize  Efficiency  Minimize  Cost 

Maximize  Effectiveness  Minimize  Cost 

Maximize  "Utility"  No  Constraint 

You  should  document  your  selection  in  the  Media  Selection 
Planning  Worksheet  (see  Table  3-5). 

Table  2-6  also  lists  the  criteria  that  must  be  selected  to 
support  each  objective-constraint  combination.  Make  sure 
you  have  selected  a  set  of  criteria  which  can  support  the 
type  of  assignment  and  objective-constraint  combination  you 
desire. 

Run  Media  Selection  Program;  At  this  point,  you  are  ready 
to  run  the  program.  The  menus  and  frames  for  running  the 
Media  Selection  program  are  described  in  Section  6.  Don't 

forget  to  bring  the  completed  Media  Selection  program 
worksheet  (Table  2-3)  and  task  data  worksheets  (Table  2-4) 
with  you  when  you  decide  to  run  the  program.  Procedures  for 
evaluating  the  results  of  the  Media  Selection  Program  are 
listed  in  Section  5. 


SECTION  3  -  MENUS  AND  FRAMES 


This  section  provides  a  detailed  description  of  the  menus 
and  frames  in  the  Media  Selection  Program.  Table  3-1 
describes  the  individual  menus  and  frames.  An  overview  of 
the  logic  among  these  menus  and  frames  is  presented  in 
Figure  3-1.  A  detailed  description  of  this  logic  is 
presented  in  Figure  3-2. 

In  the  sections  which  follow,  detailed  descriptions  of  each 
individual  menu  or  frame  are  provided.  These  descriptions 
list  (a)  the  purpose  of  each  menu,  ( b)  the  procedures  for 
using  it,  (c)  the  frames  which  will  follow  the  frame  when 
you  have  completed  the  action  required  in  the  frame,  and  (d) 
the  frame  where  the  you  will  go  if  you  select  the  abort 
option  which  is  available  in  most  frames.  In  addition,  an 
example  of  each  frame  is  provided. 


Figure  3-2.  Detailed  Description  of  Logic  in  Media  Selection  Program 


3.1  INTRODUCTORY  FRAME  (MS-1) 

Description.  This  is  the  first  frame  you  will  see  in  the 
Media  Selection  Program.  The  purpose  of  this  frame  is  to 
let  you  know  that  you  have  successfully  entered  the  Media 
Selection  Program. 

The  frame  is  listed  in  Figure  3-3.  You  have  the  option  of 
either  returning  to  the  SDT  or  continuing  with  the  Media 
Selection  Program.  To  select  an  option,  move  the  cursor 
(that  is  the  highlighted  area)  to  the  desired  option  and  hit 
Return.  (The  cursor  is  moved  by  hitting  the  up  or  down 
arrow  key . ) 

Abort  Options 

o  Exit  to  SDT-By  selecting  this  option,  you  will  return 
to  SDT  Applications  Program  Select  (Frame  SDT-5 ) 


Follow-on  Frames 


Criteria  Selection  (MS-2) 


WELCOME  TO  MEDIA  SELECTION  PROGRAM 


Return  to  SDT 
Continue 


Figure  3-3.  Introductory  Frame  (MS-1) 
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3.2  CRITERIA  SELECTION  (MS-2) 

Description.  This  menu  will  allow  you  to  select  one  of  the 
seven  sets  of  analysis  criteria  used  in  the  Media  Selection 
Program  (see  Figure  3-4).  The  seven  criteria  are  (1) 
efficiency,  (2)  effectiveness,  (3)  efficiency  and  cost,  (4) 
effectiveness  and  cost,  (5)  "utility"  with  efficiency,  (6) 
"utility"  with  effectiveness,  and  (7)  cost.  Guidelines  for 
selecting  criteria  are  provided  in  Section  2.5. 

To  select  a  set  of  criteria,  move  the  cursor  to  the  desired 
option  and  hit  Return. 

Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 


ABORT 


Efficiency 

Effectiveness 

Efficiency  and  Cost 

Effectiveness  and  Cost 

"Utility"  With  Efficiency 

"Utility"  With  Effectiveness 

Cost 
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Eigure  3-4.  Criteria  Selection  (MS-2). 


3.3  TASK  SELECTION  (MS-3) 


Description.  This  menu  allows  you  to  select  the  tasks  to  be 
included  in  the  analysis  (see  Figure  3-5).  A  maximum  of 
fifty  tasks  may  be  selected.  The  tasks  selected  should  be 
associated  with  a  course  module  or  other  logical  unit  of 
instruction. 

To  select  tasks,  move  the  cursor  up  and  down  on  the  screen 
and  press  Retur n  when  it  is  on  the  item  you  want  to  select. 
To  select  another  task  move  the  cursor  to  the  desired  item 
and  hit  Return  again.  If  all  of  the  tasks  do  not  fit  on  the 
screen  (this  is  indicated  by  the  phrase  "more"  on  the  bottom 
of  the  screen),  you  can  view  and  select  the  remaining  tasks 
by  "scrolling"  the  screen  up  (see  Section  2.2).  When  you 
have  selected  all  of  the  tasks  you  desire,  hit  No  More.  You 
may  select  all  tasks  by  moving  the  cursor  to  this  option  and 
hitting  Return.  The  All  Tasks  option  can  be  used  to  reverse 
the  current  status  (highlighted  or  non-highlighted)  of  the 
tasks  selected.  (Thus  you  can  use  this  option  to  "unselect" 
the  selected  items  which  are  highlighted  on  the  screen  or  to 
select  the  items  not  currently  highlighted). 


ABORT 


All  tasks 

PERFORM  PREFLIGHT  INSPECTION  OF  XXX  FIGHTER  AIRCRAFT 

TAXI  XXX  FIGHTER  AIRCRAFT 

PERFORM  A  SHOREBASED  VFR  TAKE  OFF 

PERFORM  A  SHOREBASED  VFR  LANDING 

OPERATE  XXX  HEADS-UP  DISPLAY 

EXECUTE  FORMATION  MANEUVERS 

COMPLETE  POST-FLIGHT  REPORT 


Figure  3-5.  Task  Selection  (MS-3). 


Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 

Follow-on  Frames 

o  If  you  have  selected  the  cost  only  option  as  your 
analysis  criteria,  you  will  go  Media  Category  Selection 
(MS-13)  . 

o  If  you  select  any  of  the  other  criteria,  you  will  go  to 
Selection  of  Psychological  Variables  (MS-4). 


3.4  SELECTION  OF  PSYCHOLOGICAL  VARIABLES  (MS-4) 


Description.  This  menu  allows  you  to  select  one  of  the 
three  categories  of  psychological  variables  (stimuli, 
responses,  and  feedback).  This  is  the  first  step  in  a  two- 
step  process  for  selecting  psychological  variables.  In 
subsequent  menus  you  will  be  able  to  select  specific 
variables  within  these  three  categories.  Guidelines  for 
selecting  categories  are  provided  in  Section  2.5. 

An  example  of  this  frame  is  provided  in  Figure  3-6.  To 
select  a  category,  move  the  category  to  the  desired  option 
and  hit  Return.  If  you  have  completed  your  selection  of 
psychological  variables,  move  the  cursor  to  the  No  More 
option  and  hit  Return. 

The  latter  action  will  take  you  to  the  Data  Review-1  Frame 
(MS-9 )  where  you  will  have  the  option  of  selecting  a 
different  set  of  tasks,  or  exiting  to  the  SDT  mode  select 
frame  (SDT-4).  (These  options  are  designed  to  let  you  deal 
with  any  missing  data  identified  during  the  selection  and 
examination  of  psychological  variables.) 

Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 

Follow-On  Frames 

o  If  you  select  the  No  More  option,  you  will  go  to  Da ta 
Review-1  (MS-9) 


i 


Select  Psychological  Variables 


ABORT 
No  More 
STIMULI 
RESPONSES 
FEEDBACK 


Figure  3-6.  Selection  of  Psychological  Variables  (MS-4) 


o 


If  you  select  the  St  imu  li  category,  you  will  go  to 
Stimulus  Variable  Selection  (MS-5) 


I) 
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3.5  STIMULUS  VARIABLE  SELECTION  (MS-5) 

This  menu  allows  you  to  select  which  of  the  twelve  variables 
which  describe  stimulus  characteristics  will  be  included  in 
the  analysis  (see  Figure  3-7).  Guidelines  for  selecting 
psychological  variables  are  provided  in  Section  2.5.  Until 
you  gain  experience  with  the  program,  it  is  recommended  that 
you  select  all  of  the  stimulus  variables. 

To  select  stimulus  variables,  move  the  cursor  up  and  down  on 
the  screen  and  press  Return  when  it  is  on  the  item  you  want 
to  select.  To  select  another  variable  move  the  cursor  to 
the  desired  item  and  hit  Return  again.  When  you  have 
selected  all  of  the  variables  you  desire,  hit  No  More.  You 
may  select  all  variables  by  moving  the  cursor  to  this  option 
and  hitting  Return.  The  All  Attributes  option  can  be  used 


to  reverse 

the 

current  sta 

tus 

(highlighted  or  non-high- 

lighted)  of 

all 

variables  on 

the 

screen.  Thus, 

you  can 

use 

this  option 

to 

"unselect" 

the 

selected  items 

which 

are 

highlighted 

on 

the  screen 

or 

to  select  the 

items 

not 

currently  highlighted. 

Once  you  have  selected  all  of  the  stimuli  variables  you 
desire  (this  is  indicated  by  hitting  the  No  More  Attributes 
option)  a  message  will  appear  on  the  right  hand  side  of  the 
screen  asking  you  if  you  want  to  view  the  scores  of  the 
tasks  you  selected  in  Task  Selection  (MS-3)  on  the  stimulus 
variables  you  have  just  selected  in  this  menu.  If  you  hit 
yes,  you  will  go  to  View  Data  for  Psychological  Variables 
( M S- 8 )  where  the  scores  of  each  task  on  the  psychological 
variables  will  be  displayed. 


3-lb 


Psychological  variable:  Stimuli. 
Select  Attributes: 

-  ABORT 

-  All  Attributes 
MEDIUM 
VISUAL  FORM 
VISUAL  MOVEMENT 
VISUAL  SPECTRUM 
VISUAL  SCALE 
VISUAL  CONTRAST 
AUDIO  SOURCES 

AUDIO  STIMULI  INTENSITY 
STIMULI  PRESENTATION 
STIMULI  PRESENTATION  RATE 
NUMBER  OF  CHANNELS 
STIMULI  DISTRIBUTION 


Figure  3-7.  Stimulus  Variable  Selection  (MS 
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If  you  hit  _no,  the  program  will  check  for  missing  data  for 
each  task  and  indicate  where  data  is  missing  by  displaying 
the  phrase  Missing  Data  beside  each  stimulus  variable.  If 
there  is  no  data  missing,  no  such  message  will  be  displayed. 
As  the  program  performs  this  check  for  each  task,  the  name 
of  the  task  will  be  displayed  at  the  top  of  the  frame.  Once 
the  program  has  checked  for  missing  data  you  will  go  to 
Select  Psychological  Variables  (MS-4),  where  you  will  have 
the  option  of  selecting  another  category  of  psychological 
variables  or  continuing  with  the  analysis. 

Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 


Follow-on  Frames 

o  If  you  select  the  option  for  viewing  data  you  will  go 
to  the  View  Data  for  Psychological  variables  (MS-8). 


3.6  RESPONSE  VARIABLE  SELECTION  (MS-6) 

This  menu  allows  you  to  select  which  of  the  six  variables 
describing  response  characteristics  will  be  included  in  the 
analysis  (see  Figure  3-8).  Guidelines  for  selecting  psycho¬ 
logical  variables  are  provided  in  Section  2.5.  Until  you 
gain  experience  with  the  program,  it  is  recommended  that  you 
select  all  of  the  response  variables. 

To  select  a  response  variable,  move  the  cursor  up  and  down 
on  the  screen  and  press  Return  when  it  is  on  the  item  you 
want  to  select.  To  select  another  variable  move  the  cursor 
to  the  desired  item  and  hit  Return  again.  When  you  have 
selected  all  of  the  variables  you  desire,  hit  No  More.  You 
may  select  all  variables  by  moving  the  cursor  to  this  option 
and  hitting  Return.  The  All  Attributes  option  can  be  used 
to  reverse  the  current  status  (highlighted  or  non-high- 
lighted)  of  the  variables  selected.  Thus,  you  can  use  this 
option  to  "unselect"  the  selected  items  which  are  high¬ 
lighted  on  the  screen  or  to  select  the  items  not  currently 
highlighted. 

Once  you  have  selected  all  of  the  response  variables  you 
desire  (this  is  indicated  by  hitting  the  No  More  Attributes 
option)  a  message  will  appear  on  the  right  hand  side  of  the 
screen  asking  you  if  you  want  to  view  the  scores  of  tasks 
you  selected  in  Task  Selection  (MS-3)  on  the  response 
variables  you  have  just  selected  in  this  menu. 

If  you  hit  yes,  you  will  go  to  View  Data  for  Psychological 
Variables  (MS-8)  where  the  scores  of  each  task  on  the 
psychological  variables  will  be  displayed. 


Responses. 


55 

l 


Select  attributes 


-  ABORT 


-  All  Attributes 


RESPONSE  MODE 


INTENSITY  OF  RESPONSE 


RESPONSE  IMPLEMENTATION 


REQUIRED  RESPONSE  RATE 


#  RESPONSE  CHANNELS 


RESPONSE  DISTRIBUTION 


Figure  3-8.  Response  Variable  Selection  (MS-6). 
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If  you  hit  _no,  the  program  will  still  check  for  missing  data 
for  each  task  and  indicate  where  data  is  missing  by 
displaying  the  phrase  Missing  Data  beside  each  response 
variable.  If  there  is  no  missing  data  no  such  message  will 
be  displayed.  As  the  program  performs  this  check  for  each 
task,  the  name  of  the  task  will  be  displayed  at  the  top  of 
the  frame. 

Once  the  program  has  checked  for  missing  data,  you  will  go 
to  Select  Psychological  Variables  (MS-4),  where  you  will 
have  the  option  of  selecting  another  category  of 
psychological  variables  or  continuing  with  the  analysis. 


Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 

Follow-On  Frames 

o  If  you  select  the  option  for  viewing  data  you  will  go 
to  View  Data  for  Psychological  Variables  (MS-8). 

o  If  you  do  not  select  the  option  for  viewing  data  you 
will  go  back  to  Selection  of  Psychological  Variables 
(MS-4 )  where  you  can  select  another  category  of 
psychological  variables  or  go  on  to  the  next  step  in 
the  analysis. 


3.7  FEEDBACK  VARIABLE  SELECTION  (MS-7) 


This  menu  allows  you  to  select  which  of  the  twelve  variables 
describing  feedback  characteristics  will  be  included  in  the 
analysis  (see  Figure  3-9).  Guidelines  for  selecting  psycho¬ 
logical  variables  are  provided  in  Section  2.5.  Until  you 
have  gained  experience  with  the  program,  it  is  recommended 
that  you  select  all  of  the  feedback  variables. 

To  select  a  feedback  variable,  move  the  cursor  up  and  down 
on  the  screen  and  press  Return  when  it  is  on  the  item  you 
want  to  select.  To  select  another,  move  the  cursor  to  the 
desired  item  and  hit  Return  again.  When  you  have  selected 
all  of  the  variables  you  desire,  hit  No  More.  You  may  select 
all  variables  by  moving  the  cursor  to  this  option  and 
hitting  Return.  The  All  Attributes  option  can  be  used  to 
reverse  the  current  status  (highlighted  or  non-highlighted) 
of  the  variables  selected.  Thus,  you  can  use  this  option  to 
"unselect"  the  selected  items  which  are  highlighted  on  the 
screen  or  to  select  the  items  not  currently  highlighted. 

Once  you  have  selected  all  of  the  feedback  variables  you 
desire  (this  is  indicated  by  hitting  the  No  More  Attributes 
option)  a  message  will  appear  on  the  right  hand  side  of  the 
screen  asking  you  if  you  want  to  view  the  scores  of  tasks 
you  selected  in  Task  Selection  (MS-3)  on  the  feedback 
variables  you  have  just  selected  in  this  menu. 

If  you  hit  yes ,  you  will  go  to  View  Data  for  Psychological 
Variables  (MS-8)  where  the  scores  of  each  task  on  the 
psychological  variable  will  be  displayed. 


Psychological  Variable:  Feedback, 


Select  Attributes 


-  ABORT 


-  ALL  ATTRIBUTES 


FEEDBACK  MEDIUM 


FEEDBACK  SOURCE 


FEEDBACK  TYPE 


FEEDBACK  DISTRIBUTION 


Figure  3-9.  Feedback  Variable  Selection  (MS-7) 
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If  you  hit  _no,  the  program  will  still  check  for  missing  data 
for  each  task  and  indicate  where  data  is  missing  by 
displaying  the  phrase  "missing  data"  beside  each  feedback 
variable.  If  there  is  no  data  missing,  no  such  message  will 
be  displayed.  As  it  performs  this  check  for  each  task,  the 
name  of  the  task  will  be  displayed  at  the  top  of  the  frame. 

Once  the  program  has  checked  for  missing  data,  you  will  go 
to  Select  Psychological  Variables  (MS-4),  where  you  will 
have  the  option  of  selecting  another  category  of 
psychological  variable  or  continuing  with  the  analysis. 

Abort  Options 

o  SDT  Applico cions  Program  Select  (SDT-5) 

Follow-On  Frames 


o  If  you  select  the  option  for  viewing  data  you  will  go 
to  View  Data  for  Psychological  Variables  (MS-8). 

If  you  do  not  select  the  option  for  viewing  data,  you 
will  go  to  Select  Psychological  Variables  (MS-4)  where 


o 


you  can  select  another  category  of  psychological 
variables  or  go  on  to  the  next  step  in  the  analysis. 


3.8  VIEW  DATA  FOR  PSYCHOLOGICAL  VARIABLES  (MS-8) 


If  you  have  selected  the  option  for  viewing  data,  in  the 
Stimulus  Variable  Selection  (MS-SO,  Response  Variable 
Selection  (MS-6),  or  Feedback  Variable  Selection  (MS-7) 
frames,  the  data  for  the  variables  you  have  selected  will 
appear  on  the  screen  to  the  right  of  the  variables  names. 
An  example  of  this  frame  is  presented  in  Figure  3-10.  The 
data  will  begin  appearing  immediately  after  you  hit  the  yes 
option  for  viewing  data  in  the  Stimulus  Variable  Selection 
(MS-S ) ,  Response  Variable  Selection  (MS-6),  or  Feedback 
Variable  Selection  (MS-7)  Frame.  Data  for  one  task  will 
appear  on  the  screen,  then  data  for  the  next  task  will 
appear  on  the  screen,  etc.  until  data  for  all  the  tasks  you 
have  selected  has  been  displayed.  The  title  of  each  task 
will  be  displayed  at  the  top  of  the  screen.  To  move  to  the 
next  task,  you  must  hit  Cont i nue .  If  there  is  data  missing 
for  any  of  the  tasks,  the  term  Missing  Data  will  appear  on 
the  screen  beside  each  variable  with  data  missing. 


You  can  abort  the  viewing  of  data 
option.  If  you  select  this  option, 
search  for  missing  data  for  all  of 
individual  data  elements  for  each 
displayed  on  the  screen. 


by  hitting  the  Abort 
the  program  will  still 
the  selected  tasks  but 
variable  will  not  be 


After  all  the  data  has  been  displayed  you  will  go  back  to 
Select  Psychological  Variables  (MS-4)  where  you  will  have 
the  option  of  selecting  another  type  of  psychological 
variable,  or  continuing  with  the  analysis. 


Psychological  Variable:  Feedback  for  Task  Remove  and  Replace 


AN  RM/Q15. 

Select  Attributes: 

-  ABORT 

-  ALL  ATTRIBUTES 
FEEDBACK  MEDIUM 
FEEDBACK  SOURCE 
FEEDBACK  TYPE 
FEEDBACK  DISTRIBUTION 


SDT  VALUE: 

VISUAL 

INTRINSIC 

RESPONSE  CORRECTNESS 
INDIVIDUAL 


-  Abort  viewing  these  values 

-  Continue 


Figure  3-10.  View  Data  for  Psy.  Variables  (MS-8). 
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Abort  Options 


o 


If  you  select 
for  missing 
Psycholoq  ical 


the  Abort  option, 
data  and  then 
Variable  (MS-4  ) 


the  program  will  search 
return  you  to  Select 


Follow-On  Frames 


o  Select  Psychological  Variables  (MS-4) 
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3.9  DATA  REVIEW- 1  (MS- 9) 


This  purpose  of  this  frame  is  to  allow  you  to  deal  with  any 
missing  data  for  psychological  variables  you  selected  and 
examined  in  Stimulus  Variable  Selection  (MS-5),  Response 
Variable  Selection  (MS-6),  Feedback  Variable  Selection  (MS- 
_7_)  and  View  Data  for  Psychological  Variables  (MS-8). 

The  frame  provides  you  with  three  options  (1)  continuing 
with  the  analysis,  (2)  selecting  a  different  set  of  tasks, 
or  (3)  exiting  the  program  to  enter  the  missing  data  into 
the  SDT. 

If  you  have  data  missing,  you  should  either  select  a  new  set 
of  tasks  for  which  you  are  sure  there  is  data,  or  you  should 
exit  and  enter  the  SDT  where  you  can  enter  data  for  the 
tasks  with  data  missing.  Only  if  there  is  no  missing  data 
should  you  select  the  continue  option.  If  you  continue,  the 
program  will  assign  a  value  of  zero  to  each  psychological 
variable  with  missing  data. 

Abort  Options 

o  If  you  select  the  Abort  option  you  will  go  back  to  the 
SDT  Applications  Program  Select  (SDT-5).  If  you  select 
this  option,  all  of  your  work  from  the  current  session 
of  the  Media  Selection  Program  will  be  destroyed. 

o  If  you  select  the  option  for  exiting  to  enter  missing 
data  into  the  SDT,  you  will  go  back  to  the  SDT 
Introduction  (SDT-1). 


If  you  would  like  to  pick  a  different  set  of  tasks,  please 
select  the  appropriate  option. 


-ABORT 


-Continue 


-Different  tasks 


-Exit  to  enter  missing  data  into  SDT 


Figure  3-11.  Data  Review-1  (MS-9] 
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3.10  CRITICALLY  WEIGHTS  (MS-10) 


This  frame  allows  you  to  enter  weights  for  the  eight 
variables  used  to  assess  task  criticality  (task  frequency, 
percent  members  performing,  percent  time  performing,  task 
delay  tolerance,  consequences  of  inadequate  performance, 
task  learning  difficulty,  probability  of  deficient 
performance,  and  time  between  entry  and  performance). 

The  frame  will  begin  by  asking  you  whether  you  would  like  to 
use  equal  weights  or  not.  Unless  you  are  an  experienced 
user,  you  should  select  the  equal  weights  option.  If  you 
select  the  equal  weights  option,  the  program  will  assign 
equal  weights  to  all  eight  variables  and  you  will  be  asked 
if  you  want  to  review  the  scores  of  the  tasks  on  the  eight 
psychological  variables. 

If  you  select  the  option  for  inputting  weights,  spaces  will 
appear  in  the  screen  for  entering  weights  for  each  variable 
(see  Figure  3-12).  To  enter  a  weight,  use  your  keyboard  to 
type  in  the  weights  .  You  must  input  a  number  between  0  and 
100.  The  program  will  automatically  convert  your  input  to 
fractional  weights.  For  example,  if  you  type  in  50  it  will 
assign  a  weight  of  .50.  If  you  make  a  mistake  you  can 
correct  it,  by  moving  the  cursor  back  and  retyping  the 
information  before  you  hit  Return.  You  can  eliminate  a 
criticality  variable  from  the  analysis  by  assigning  it  a 
value  of  0. 

The  weights  you  assign  must  sum  to  100.  If  you  assign  a  set 
of  weights  and  they  do  not  sum  to  100,  you  will  get  an 
error  message  and  the  program  will  ask  you  to  enter  the 
weights  again,  starting  with  the  first  criticality  variable 


— 7  l*“* 


Weights  for  criticality  variables  must  be  between  0-100  and 
sum  to  100. 


Please  enter  weight  for 
PER  CENT  PERFORMING 


4  _ 

THEN  PRESS  RETURN 
(MAKE  NO  ENTRY  TO  ABORT) 


Figure  3-12.  Criticality  Weights  (MS-10). 


-  Percent  Performing.  After  you  have  entered  the  weights 
the  program  will  give  you  the  option  of  continuing  or 
reentering  the  weights.  If  you  hit  continue  a  message  will 
appear  on  the  screen  asking  if  you  would  like  to  view  the 
criticality  data  for  the  tasks  you  have  selected.  If  you 
select  the  yes  option  for  viewing  criticality  data  you  will 
go  to  View  Criticality  Data  (MS-11).  If  you  select  the  no 
option  for  viewing  criticality  data,  and  there  is  missing 
criticality  data  you  will  go  to  Data  Review  -  2  (MS-12).  If 
you  select  the  no  option  for  viewing  criticality  and  there 
is  no  missing  data,  the  program  will  calculate  criticality 
scores.  This  will  be  indicated  by  the  message  computing  .  . 

.  please  wait.  After  the  scores  are  calculated  you  will  go 
to  Media  Category  Selection  (MS-13) 


Abort  Options 

o  If  you  make  no  entry  for  any  particular  criticality 

variable,  you  will  go  to  the  form  for  inputting  data  on 
the  next  criticality  variable. 

o  If  you  select  the  option  for  aborting  criticality 

values  which  appears  after  all  the  weights  have  been 
entered,  you  will  return  to  the  form  for  inputting  data 
for  the  first  criticality  variable. 

Follow-on  Frames 

o  If  you  select  the  option  for  viewing  the  criticality 
data,  you  will  go  to  View  Criticality  Data  (MS-4/ 


If  you  do  not  select  the  option  for  viewing  criticality 
data  and  there  is  missing  criticality  data,  you  will  go 
to  Data  Review- 2  (MS-12) 


If  you  do  not  select  the  option  for  viewing  criticality 
data  and  there  is  no  missing  criticality  data,  the 
program  will  calculate  criticality  scores  and  then  you 
will  go  to  Media  Category  Selection  (MS-13) 
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3.11  VIEW  CRITICALITY  DATA  (MS-11) 


During  this  frame,  you  can  view  the  criticality  data  for 
each  of  the  tasks  you  selected.  Data  will  be  displayed  for 
each  criticality  variable.  An  example  of  this  frame  is 
provided  in  Figure  3-13.  If  there  is  data  missing,  the 

message  No  Data  will  be  displayed  beside  the  variable  with 
data  missing.  When  you  have  completed  your  review  of  the 
criticality  data  for  the  first  task  move  the  cursor  to 
Continue  and  hit  Return.  The  data  for  the  second  task  you 
have  selected  will  appear  on  the  screen,  followed  by  the 

data  for  the  next,  etc.,  until  the  data  for  all  tasks  has 

been  displayed. 

Abort  Options 

o  If  you  abort  viewing  criticality  values,  the  program 
will  compute  criticality  scores.  If  there  is  no 
missing  data,  you  will  then  go  to  Media  Category 

Selection  (MS-13).  If  there  is  missing  data,  you  will 
go  to  Data  Review  2  (MS-12) 

Follow-on  Frames 


o  If  there  is  missing  data,  you  will  go  to  Data  Review  2 
(MS-12) 

o  If  there  is  no  missing  data,  the  program  will  compute 
criticality  scores  and  you  will  then  go  to  Media 
Category  Selection  (MS-13) 
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REPLACE  AN  RM/Q15  -  Criticality  Values 


PER  CENT  PERFORMING  58 
PER  CENT  TIME  PERFORMING  3 
CONSEQ.  INADEQ.  PERFORM.  3 
TASK  DELAY  TOLERANCE  3 
TASK  LEARNING  DIFFICULTY  3 
PROB.  OF  DIFICIENT  PERFORMANCE  3 
TIME  FROM  ENTRY  TO  PRFRM  3 


Abort  viewing  criticality  values 
Continue 


Figure  3-13.  View  Criticality  Data  (MS-11) 


3.12  DATA  REVIEW  -2  (MS-12) 

The  purpose  of  this  frame  is  to  allow  you  to  deal  with  any 
missing  data  for  the  criticality  variables. 

The  frame  is  displayed  in  Figure  3-14.  The  frame  provides 
you  with  three  options:  (1)  continuing  with  the  analysis, 

(2)  selecting  a  different  set  of  tasks,  or  (3)  exiting  to 
enter  the  missing  data  into  the  SDT. 

If  you  have  data  missing  you  should  either  select  a  new  set 
of  tasks  for  which  you  are  sure  there  is  data,  or  you  should 
exit  and  enter  the  SDT  to  input  data  for  the  tasks  you  have 
selected. 

Only  if  there  is  no  missing  data  should  you  select  the 
continue  option. 

Abort  Options 

o  Exit  to  enter  missing  data  into  SDT.  By  selecting  this 
option  you  will  go  back  to  the  SDT  Introduction  Frame 
( S DT- 1 )  .  If  you  select  this  option  all  of  your  work 

from  the  current  session  of  the  Media  Selection  Program 
will  be  destroyed. 

Follow-on  Frames 


o  If  you  select  the  option  for  selecting  different  tasks 
you  will  go  to  Task  Selection  (MS-3). 


3-44 


There  is  criticality  data  missing.  Please  select  the  appropria 
option . 

-  Different  Tasks 

-  Exit  to  enter  missing  data  into  SDT 


Figure  3-14.  Data  Review-2  (MS-12). 


3.13  MEDIA  CATEGORY  SELECTION  (MS-13) 

This  frame  allows  you  to  select  which  of  the  twenty  nine 
media  categories  hard-coded  in  the  program  will  be  used  in 
the  analysis.  A  listing  of  the  media  categories  is  provided 
in  Tcible  3-2.  The  frame  for  selecting  media  categories  is 
presented  in  Figure  3-15.  Guidelines  for  selecting  media 
categories  are  provided  in  Section  2.5.  Unless  you  are 
familiar  with  the  program,  you  should  select  all  media 
categ ories. 

To  select  media  categories,  move  the  cursor  up  and  down  on 
the  screen  and  press  Return  when  it  is  on  the  media  category 
you  want  to  select.  To  select  another  category  move  the 
cursor  to  the  desired  item  and  hit  Return  again.  Since  all 
of  the  categories  do  not  fit  on  the  screen  (this  is 
indicated  by  the  phrase  more  on  the  bottom  of  the  screen), 
you  can  view  and  select  the  remaining  categories  by 
"scrolling"  the  screen  up  (see  Section  2.2).  When  you  have 
selected  all  of  the  categories  you  desire,  hit  No  More.  You 
may  select  All  categories  by  moving  the  cursor  to  this 
option  and  hitting  Return.  The  All  Media  Categories  option 
can  be  used  to  reverse  the  current  status  (highlighted  or 
non-highlighted)  of  the  categories  selected.  Thus,  you  can 
use  this  option  to  "unselect"  the  selected  items  which  are 
highlighted  on  the  screen  or  to  select  the  items  not 
currently  highlighted. 

Abort  Options 

SDT  Applications  Program  Select  (SDT-5) 
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O 


Please  select  built-in  media  categories  for  analysis 


& 


ABORT 

All  Media  Categories 
Prnt  Ins  w/o  Fd 
Prnt  In  w/oral  Fd 
Prnt  In  w/oral  Fd 
Prnt  In  w/writ  Fd 
Prnt  -  Ref  Material 
Image  Proj  -  Still 
Passive  Audio 
Active  Audio 
AV  -  Still  w/Fd 
AV  -  Still  w/o  Fd 
AV  -  Motion  w/o  Fd 
AV  -  Mot  w/o  Fd 
Stat  Disp  w/o  Fd 
Dyn  Displ  w/o  Fd 
Phys  Train  -  inter 
Phys  Train  -  audio 
Phys  Train  -  visual 
-  more  - 


Figure  3-15.  Media  Category  Selection  (M-13) 
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Table  3-2.  Media  Categories. 


MEDIA  CATEGORY 

PRINT  INSTRUCTIONS  W/0  FEEDBACK 

PRINT  INSTRUCTIONS  WITH  ORAL  FEEDBACK 

PRINT  INSTRUCTION  -  WITH  WRITTEN  FEEDBACK 

PRINT  -  REFERENCE  MATERIAL 

IMAGE  PROJECTION  -  STILL 

PASSIVE  AUDIO 

ACTIVE  AUDIO 

AUDIOVISUAL  -  STILL  WITH  FEEDBACK 
AUDIOVISUAL  -  STILL  WITHOUT  FEEDBACK 
AUDIOVISUAL  -  MOTION  WITH  FEEDBACK 
AUDIOVISUAL  -  MOTION  WITHOUT  FEEDBACK 
STATIC  DISPLAY  WITHOUT  FEEDBACK 
DYNAMIC  DISPLAY  WITOOUT  FEEDBACK 
PHYSIOLOGICAL  TRAINER  -  INTERNAL 
PHYSIOLOGICAL  TRAINER  -  AUDIO 
PHYSIOLOGICAL  TRAINER  -  VISUAL 
SYMBOLIC  SIMULATION  WITH  FEEDBACK 
TRAINER 
SIMULATOR 

OPERATIONAL  EQUIPMENT 
OPERATIONAL  EQUIPMENT  WITH  FEEDBACK 
VISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 
VISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 
VISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 
VISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 
INSTRUCTOR 
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Follow-on  Frames 


o  If  you  have  selected  cost  only  as  your  analysis 
criterion  the  program  now  has  all  the  information 
needed  to  make  task-media  assignments.  The  program 
will  print  out  the  assignments  and  you  will  go  to 
Select  Rerun  Option  (MS-23). 

o  If  you  did  not  select  the  cost  only  criterion  option, 
you  will  go  to  Computation  of  Efficiency/Effectiveness 


(MS-14 ) 


3.14  COMPUTATION  OF  EFF IC I ENC Y/EFF ECTI VENESS  (MS-14) 

During  this  frame  the  message  Computi ng-please  wait  will  be 
displayed  on  the  screen  (see  Figure  3-16).  This  message 
will  remain  on  the  screen  until  the  program  has  calculated 
all  efficiency  and  effectiveness  scores. 

Abort  Options 


None . 


Follow-on  Frames 


Option  for  View  Efficiency/Effectiveness  Scores  (MS-15) 


Sam 


3.15  OPTION  FOR  VIEWING  EFFICIENCY/EFFECTIVENESS  SCORES 
(MS-15 ) 

During  this  frame,  you  can  select  the  option  for  examining  a 
histogram  of  efficiency  scores  on  your  computer  screen  (see 
Figure  3-17). 

To  select  an  option,  move  the  cursor  to  the  desired  option 
and  hit  Return. 

Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 

Follow-on  Frames 

o  If  you  select  the  option  for  viewing  graphs  of 
efficiency  scores  on  your  computer  screen  you  will  go 
to  Selection  of  Media  Categories  for  Display  (MS-16). 


)  If  you  do  not  select  the  option  for  viewing  efficiency 
scores  and  have  not  selected  "utility"  as  an  analysis 
criterion,  you  will  go  to  Print  Criteria  Scores  (MS- 
19). 

3  If  you  do  not  select  the  option  for  viewing  efficiency 
scores  and  you  have  selected  "utility"  as  an  analysis 
criterion,  you  will  go  to  "Utility"  Weights  (MS-18). 
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Would  you  like  to  view  bar  graphs  for  efficiency  scores? 


-  yes 


-  no 


Figure  3-17.  Option  for  Viewing  Efficiency/ 
Effectiveness  Scores  (MS-15) . 


3.16  SELECTION  OF  MEDIA  CATEGORIES  FOR  DISPLAY 

During  this  frame,  you  can  select  which  of  the  media 
categories  will  be  included  in  the  graphical  displays  of 
efficiency  scores  on  your  computer  screen  (see  Figure  3- 
18).  No  more  than  ten  media  categories  may  be  selected  tor 
display . 

To  select  media  categories,  move  the  cursor  up  and  down  on 
the  screen  and  press  Return  when  it  is  on  the  item  you  wan1- 
to  select.  To  select  another  category  move  the  cursor  a 
the  desired  item  and  hit  Retur  n  again.  When  you  h 
selected  all  of  the  categories  you  desire,  hit  No  More.  u 
may  select  All  Media  Categories  by  moving  the  cursor  to  t  ; 
option  and  hitting  Retur n.  The  All  Media  Categories  option 
can  also  be  used  to  reverse  the  current  status  (highlighted 
or  non-highlighted)  of  the  categories  selected.  Thus,  you 
can  use  this  option  to  "unselect"  the  selected  items  which 
are  highlighted  on  the  screen  or  to  select  the  items  not 
currently  highlighted. 

Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 

Follow-on  Frames 


o 


Graphical  Screen  Display  (MS-15) 


Please  select  media  categories  for  bar  graph. 


-  ABORT 

Prnt  Ins  w/o  Fd 
Prnt  Ins  w/oral  Fd 
Prnt  Ins  w/writ  Fd 


Figure  3-18.  Selection  of  Media  Categories  For  Display  (MS-16) 

F: 


3.17  GRAPHICAL  SCREEN  DISPLAY  (MS-17) 

During  this  frame,  graphs  displaying  the  efficiency  of  each 
selected  media  category  will  be  displayed  on  the  computer 
screen.  An  example  of  one  of  these  graphs  is  displayed  in 
Figure  3-19.  The  efficiency  scores  displayed  in  this 
graphic  will  range  from  a  minimum  score  of  1  to  a  maximum 
score  of  100.  After  completing  your  examination  of  a  graph, 
hit  Return  and  bar  graph  for  next  task  will  appear  on  the 
screen,  until  graphs  have  been  displayed  for  each  of  the 
tasks  you  selected.  You  may  abort  out  of  the  graph  displays 
by  hitting  Abort .  After  the  last  graph  has  been  displayed, 
the  program  will  ask  you  if  you  want  to  choose  a  different 
set  of  media  categories.  If  you  select  this  option  you  will 
return  to  Selection  of  Media  Categories,  For  Display  (MS- 
16).  If  you  do  not  select  this  option  you  will  go  to 
Utility  Weights  (MS-18)  or  Print  Criteria  (MS-19). 

Abort  Options 

o  If  you  selected  a  criteria  involving  effectiveness  you 
will  go  to  Utility  Weights  (MS-18) 

o  If  you  did  not  select  a  criteria  involving  effective¬ 
ness  you  will  go  to  Print  Criteria  (MS-19) 

Follow-on  Frames 


o 


If  you  select  the  option  for  choosing  new  media 


Replace  AN  RM/Q15 


Print  Ins  w/o  fd 
Print  Ins  w/oral  fd 


0  10  20  30  40  50  60  70 


Efficiency 


—Abort 

—Continue 


Figure  3-19.  Graphical  Screen  Display  (MS-17). 


3.18  "UTILITY"  WEIGHTS  (MS-18) 


This  frame  allows  you  to  input  weights  for  the  two  criteria 
used  in  the  construction  of  an  overall  "utility"  measure  (a 
detailed  description  of  the  algorithm  used  to  compute 
"utility"  is  provided  in  Appendix  A). 

An  example  of  the  frame  is  presented  in  Figure  3-20.  To 
enter  a  weight  for  a  criterion,  type  in  an  integer  between  0 
and  100.  The  program  will  automatically  convert  your  input 
to  fractional  weights.  For  example,  if  you  type  in  50  the 
program  will  assign  the  variable  a  weight  of  .5.  If  you  make 
a  mistake,  you  can  correct  it  by  moving  the  cursor  back  and 
retyping  the  information.  (This  must  be  done  before  hitting 
Return.  )  The  weights  must  sum  to  100.  Until  you  have 
gained  experience  with  the  program,  it  is  recommended  that 
you  use  equal  weights  of  50  for  both  criteria. 


Once  you  have  entered  the  weight  for  the  first  variable, 
space  will  be  provided  for  entering  the  weight  for  the 
second  variable.  When  you  have  entered  the  second  weight, 
the  program  will  ask  you  if  you  want  to  stay  with  the 
weights  you  have  just  entered  or  select  a  new  set  of 
weights.  If  you  select  the  option  for  choosing  a  new  set  of 
weights,  the  form  for  entering  the  first  utility  weight  will 
appear  on  the  screen.  If  you  do  not  select  this  option,  you 
will  go  to  Print  Criteria  (MS-19). 

Abort  Options 


If  you  select  the  Abort  option  which  will  appear  after 
all  the  weights  have  been  entered  you  will  go  back  to 
the  form  for  inputting  weights  on  the  first  variable. 


<» 


Weights  for  effectiveness  and  cost  must  be  between  0-100  and 
sum  to  100. 


Please  enter  weight  for 
Effectiveness 

r 

THEN  PRESS  RETURN 
(MAKE  NO  ENTRY  TO  ABORT) 


Figure  3-20.  Utility  Weights  (MS-18). 


3.19  PRINT  CRITERIA  (MS-19) 


This  frame  will  allow  you  to  print  one  of  the  three 
individual  analysis  criteria  calculated  by  the  program  (see 
Figure  3-21).  The  three  options  are  (1)  print  efficiency, 
(2)  print  effectiveness  (3)  print  "utility." 

To  select  an  option,  move  the  cursor  to  the  desired  option 
and  hit  Return.  The  scores  will  then  be  printed  out  for 
that  option.  The  Print  Criteria  frame  will  appear  again 
giving  you  the  option  of  obtaining  printed  output  for 
another  criterion,  or  leaving  the  print  option  (this  is 
indicated  by  hitting  No  More . )  An  example  printout  is 
displayed  in  Thble  3-3.  Each  row  represents  one  task,  and 
each  column  represents  a  media  category  A  key  for  inter¬ 
preting  the  media  category  numbers  is  displayed  in  Table  3- 


Once  you  either  have  selected  the  No  More  option  indicating 
that  you  are  finished  printing  criteria  scores  or  the 
Continue  option  indicating  that  you  want  to  skip  the 
printing  of  scores,  a  message  will  appear  on  the  screen 
asking  you  if  the  results  you  obtained  were  satisfactory. 
If  you  select  the  yes  option,  you  will  then  go  to  the  first 
frame  in  the  task  assignment  process  Selection  of  Objective, 
MS-20.  If  you  select  jto,  you  will  go  to  Criteria  Selection 
( M S- 2 ) ,  where  you  can  start  your  analysis  over  again. 

Abort  Option 


None 
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Table  3-3.  Example  Printout  of  Criteria  Scores 


Table  3-4.  Media  Categories. 


CAT 

GROUP 

# 

1 

1.1 

2 

1.2 

3 

1.3 

4 

1.4 

5 

1.5 

6 

2.1 

7 

2.2 

8 

3.1 

9 

3J2 

io 

X3 

11 

3.4 

12 

4.1 

13 

4*2 

14 

4.3 

15 

4.4 

16 

4.5 

17 

4.6 

18 

4.7 

19 

4.8 

20 

4.9 

21 

4.10 

22 

5.1 

23 

5.2 

24 

5.3 

29 

5.4 

26 

S4i 

27 

5.6 

28 

5.7 

29 

5.8 

30 

6.1 

MEDIA  CATEGORY 

PRINT  INSTRUCTIONS  W/0  FEEDBACK 

PRINT  INSTRUCTIONS  WITH  ORAL  FEEDBACK 

PRINT  INSTRUCTION  -  WITH  WRITTEN  FEED8ACK 

PRINT  -  REFERENCE  MATERIAL 

IMAGE  PROJECTION  -  STILL 

PASSIVE  AUOIO 

ACTIVE  AUDIO 

AUDIOVISUAL  -  STILL  WITH  FEEDBACK 
AUDIOVISUAL  -  STILL  WITHOUT  FEEDBACK 
AUDIOVISUAL  -  MOTION  WITH  FEEDBACK 
AUDIOVISUAL  -  MOTION  WITHOUT  FEEDBACK 
STATIC  DISPLAY  WITHOUT  FEEDBACK 
DYNAMIC  DISPLAY  WITHOUT  FEEDBACK 
PHYSIOLOGICAL  TRAINER  -  INTERNAL 
PHYSIOLOGICAL  TRAINER  -  AUDIO 
PHYSIOLOGICAL  TRAINER  -  VISUAL 
SYMBOLIC  SIMULATION  WITH  FEEDBACK 
TRAINER 
SIMULATOR 

OPERATIONAL  EQUIPMENT 
OPERATIONAL  EQUIPMENT  WITH  FEEDBACK 
VISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 
VISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 
VISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 
VISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 
AUDIOVISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 
INSTRUCTOR 
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Follow-on  Frames 

o  If  you  hit  yes /  indicating  that  your  results  are  satis¬ 
factory  you  will  go  to  Selection  of  Objective  (MS-20). 


If  you  hit  _no,  indicating  that  your  results  are  not 
satisfactory,  the  program  will  ask  you  if  you  want  to 
abort  to  SDT  Applications  Program  Select  (SDT-5)  or 
select  different  criteria.  If  you  choose  different 
criteria,  you  will  go  to  Criteria  Selection  (MS-2).  If 
you  select  this  option,  all  of  your  work  from  the 
current  session  will  be  destroyed. 


3.20  SELECTION  OF  OBJECTIVE  (MS-20) 


This  frame  will  allow  you  to  select  the  objective  which  will 
guide  the  task  assignment  process.  There  are  four  possible 
objectives:  (1)  minimize  cost,  (2)  maximize  efficiency,  (3) 
maximize  effectiveness,  and  (4)  maximize  "utility."  Table 
3-5  provides  guidance  for  selecting  objectives.  This  table 
describes  the  types  of  assignments  that  will  result  from 
each  combination  of  an  objective  and  a  constraint. 
(Constraints  will  be  selected  in  the  next  frame.)  To  select 
an  objective,  examine  the  column  labeled  "assignment 
strategy,"  pick  the  type  of  assignment  that  meets  your 
needs,  and  select  the  objective  that  matches  that  assignment 
strategy.  Make  sure  that  the  objective  you  selected  is 
congruent  with  the  analysis  criteria  you  selected.  Some 
objectives  will  not  be  possible  with  certain  criteria.  The 
assignment  strategies  which  are  the  most  informative  and 
easiest  to  interpret  are  stated  in  T&ble  3-5. 

To  select  an  objective  move  the  cursor  to  the  desired 
objective  and  hit  Return. 

Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 

Follow-on  Frames 


o  Selection  of  Constraint  (MS-21) . 
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3.21  SELECTION  OF  CONSTRAINT  (MS-21) 

This  frame  allows  you  to  select  a  constraint  to  guide  the 
task  assignment  process.  To  select  a  constraint,  review 
the  assignment  strategies  listed  in  Table  3-5,  and  select  a 
constraint  which  is  appropriate  for  the  assignment  strategy 
you  selected  during  the  selection  of  an  objective  in  the 
previous  frame.  After  you  select  a  constraint,  the  program 
will  automatically  print  two  columns  listing  the  scores  on 
the  objective  and  the  constraint  variables.  This 

information  will  be  used  in  the  next  frame  to  select  a 
constraint  level.  (See  Constraint  Level,  MS-21.)  If  you  do 
not  select  a  constraint,  that  is  you  select  the  No 
Constraint  option,  the  program  will  automatically  print  the 
task-media  assignments  and  you  will  go  to  Select  Rerun 
Option  (MS-23) . 

An  example  of  this  frame  is  provided  in  Figure  3-23.  To 
select  a  constraint,  move  the  cursor  to  the  desired  item  and 
hit  Return. 

Abort  Options 

o  SDT  Applications  Program  Select  (SDT-5) 

Follow-on  Frames 

o  If  you  select  the  No  Constraint  option  you  will  go  to 
Select  Rerun  Option  (MS-23) . 


o 


If  you  select  a  constraint,  you  will  go  to  Constra int 
Level  (MS-22) . 


Please  select  constraint 

-  ABORT 

Restart  Selection 
Minimize  Cost 
Maximize  Effectiveness 
No  Constraint 


Figure  3-23.  Constraint  Selection  (MS-21) 


3.22  CONSTRAINT  LEVEL  (MS-22) 

During  this  frame,  you  will  be  required  to  enter  the  minimum 
possible  total  score  on  the  constraint  variable  that  you  are 
willing  to  accept  (or  maximum  possible  if  you  have  selected 
minimize  cost  as  you  constraint).  To  select  the  appropriate 
level,  you  must  examine  the  printout  of  scores  on  the 
objective  and  constraint  values  that  was  printed  after  your 
selected  a  constraint  in  the  previous  frame  (Selection  of 
Constraint  MS-21).  An  example  of  one  of  these  printouts  is 
presented  in  Table  3-6.  The  printout  will  list  the  best 
objective  value  obtainable  for  each  of  a  range  of  constraint 
levels.  Associated  with  each  of  the  constraint  levels 
listed  is  an  assignment  set.  An  assignment  set  is  a 
specific  assignment  of  each  task  to  exactly  one  media 
category.  The  score  of  the  objective  variable,  for  the 
assignment  set  is  determined  by  aggregating  the  scores  of 
the  individual  tasks  on  that  variable.  Similarly,  the  score 
of  the  constraint  variable  for  the  assignment  set  is 
determined  by  aggregating  the  scores  on  the  individual  tasks 
for  the  constraint  variable.  The  constraint  levels  for  the 
assignment  sets  will  be  listed  in  ascending  order  with  the 
assignment  sets  with  the  lowest  constraint  levels  listed 
first . 

To  select  a  constraint  level,  you  must  first  determine  how 
low  of  a  score  in  the  constraint  variable,  you  are  willing 
to  tolerate.  (If  you  have  selected  to  minimize  cost  as  your 
constraint  you  must  determine  how  high  of  a  score  you  are 
willing  to  tolerate.)  Table  3-7  provides  guidelines  to 
assist  you  in  making  this  decision.  If  you  are  willing  to 
accept  a  low  score,  select  a  score  that  is  about  25%  of  the 
way  between  the  minimum  and  maximum  of  the  constraint  scores 


T 


Please  enter  constraint  level: 
Numbers  must  be  between  10  and  500 

r 

THEN  PRESS  RETURN 
(MAKE  NO  ENTRY  TO  ABORT) 


Figure  3-24.  Constraint  Level  (MS-22) 


Table  3-6.  Example  of  Printout  of 
Constraint  and  Objective  Variable  Scores 


OBJECTIVE: 
CONSTRAINT : 


MAXIMIZE  EFFICIENCY 
MINIMIZE  COST 


CONSTRAINT  LEVEL 


OBJECTIVE  VALUE 


(the  minimum  and  maximum  will  be  listed  at  the  top  of  the 
frame.)  ..f  you  are  willing  to  accept  a  moderate  score  on 
the  constraint  variable  select  a  score  that  is  about  half 
way  between  these  two  scores.  If  you  are  willing  to  accept 
a  high  score,  select  a  score  that  is  75%  of  the  way  between 
the  minimum  and  maximum.  If  you  have  selected  Minimize  Cost 
as  constraint  this  logic  would  be  reversed  (see  Table  3-7). 

When  you  are  selecting  a  specific  constraint  level,  you 
should  examine  the  printout  to  select  a  constraint  score 
which  is  associated  with  a  significant  jump  in  the  score  for 
the  objective  value  which  is  close  to  your  targeted  level 
25%,  50%,  75%. 

To  enter  the  constraint  level,  simply  type  it  the  number  you 
have  selected  in  the  spaces  provided.  You  must  input  an 
integer  which  is  between  the  minimum  and  maximum  constraint 
level. 

Abort  Options 

o  Select  Rerun  Option  (MS-23) 


Follow-On  Frames 


3.23  SELECT  RETURN  OPTION  (MS-23) 


After  you  have  selected  the  constraint  level,  the  task-media 
assignment  will  automatically  be  printed.  When  this 
printing  is  complete,  the  Select  Return  Option  (MS-23)  will 
appear  on  the  screen.  (See  Figure  3-25.)  This  frame  will 
allow  you  the  option  of  (1)  exiting  to  the  SDT  where  you  can 
terminate  your  session  in  the  computer  or  use  the  SDT  data 
base  management  system,  (2)  selecting  different  criteria,  or 
(3)  selecting  different  objectives. 

To  select  an  option,  move  the  cursor  to  the  desired  item  and 
hit  Return.  An  example  of  the  Task-Media  Assignment  listing 
is  provided  in  Table  3-8. 

Abort  Options 

By  selecting  the  Abort  option,  you  can  exit  to  the  SDT 
Applications  Program  Select  (SDT-5).  If  you  select  this 
option  all  of  your  work  from  the  current  session  of  the 
Media  Selection  Program  will  be  destroyed. 

Follow-On  Frames 


o  If  you  select  the  option  for  choosing  different 
criteria  you  will  go  to  Criteria  Selection  (MS-2).  If 
you  select  this  option  all  of  your  work  from  the 
current  session  of  the  Media  Selection  Program  will  be 
destroyed. 

If  you  select  the  option  for  choosing  a  different 
objective  you  will  go  to  Selection  of  Objectives  (MS- 


o 


Select  rerun  option: 


-  Abort 

-  Select  different  criteria 

-  Select  different  objectives 


Figure  3-25.  Select  Rerun  Option  (MS- 


Example  of  Task-Media  Assignment  Listing 


SECTION  4 


EXAMPLE  APPLICATION 


This  section  presents  an  example  application  of  the  Media 
Selection  Program.  The  application  is  designed  to  provide 
first-time  users  with  hands-on  experience  in  using  the 
program.  The  example  will  allow  you  to  assign  a  set  of 
example  tasks  to  media  in  manner  that  will  maximize  training 
efficiency. 

On  the  pages  which  follow  each  specific  action  which  you 
must  take  to  run  through  the  example  is  described.  More 
specifically,  the  frames  which  you  will  encounter  during  the 
example  application  are  listed  in  sequence  and  the  specific 
actions  which  you  must  take  during  these  frames  are 
described. 

The  first  frame  you  will  encounter  during  the  example  is  the 
Introductory  Frame  (MS-1)  to  the  Media  Selection  Program. 
To  get  to  this  frame  you  must  load  the  SDT  software,  enter 
the  Applications  mode  of  the  SDT  and  select  the  Media 
Selection  Program.  Procedures  for  performing  these  actions 
are  described  in  Section  2.4. 


4.1  ACTION  1:  INTRODUCTION  FRAME  (MS-1) 

After  you  select  the  Media  Selection  Program  option  in  the 
SDT  applications  mode,  this  frame  will  appear  on  the  screen. 
Move  the  cursor  (or  highlighted  area)  to  the  Continue  with 
Program  option  and  hit  the  Return  key.  (The  cursor  is  moved 
by  hitting  the  up  or  down  arrow  keys.) 
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.it.  { 


WELCOME  TO  MEDIA  SELECTION  PROGRAM 


-  Return  to  SDT 

-  Continue 


Figure  4-1.  Introductory  Frame  (MS-1). 
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4.2  ACTION  2:  CRITERIA  SELECTION  (MS-2) 


This  frame  will  allow  you  to  select  the  criteria  that  will 
be  used  in  the  analysis.  Move  the  cursor  to  the  Ef f ic iency 
option  and  hit  Return. 


4.3  ACTION  3:  TASK  SELECTION  (MS-3) 

This  frame  will  allow  you  to  select  the  tasks  used  in  the 
analysis.  Move  the  cursor  to  the  All  tasks  option  and  hit 
Return.  Then  move  the  cursor  to  No  More  and  hit  Return. 

4.4  ACTION  4:  SELECTION  OF  PSYCHOLOGICAL  VARIABLES  MS-4) 

This  frame  will  allow  you  to  select  one  of  the  three 
categories  of  psychological  variables  (stimuli,  responses, 
and  feedbacks).  In  subsequent  frames,  you  will  select 
specific  variables  within  each  of  these  three  categories. 
(Efficiency  is  determined  by  calculating  the  match  between 
tasks  and  media  on  these  psychological  variables). 

Move  the  cursor  to  Stimuli  and  hit  Return. 

4.5  ACTION  5:  STIMULUS  VARIABLE  SELECTION  (MS-5) 

This  frame  will  allow  you  to  select  the  specific  stimulus 
variables  to  be  included  in  the  analyses.  Move  the  cursor 
to  All  Attributes  and  hit  Return.  Then  move  the  cursor  to 
No  More  and  hit  Return. 
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ABORT 


Efficiency 

Effectiveness 

Efficiency  and  Cost 

Effectiveness  and  Cost 

"Utility"  With  Efficiency 

"Utility"  With  Effectiveness 

Cost 


Figure  4-2.  Criteria  Selection  (MS-2) 


Select  tasks  for  analysis 


ABORT 
All  tasks 

PERFORM  PREFLIGHT  INSPECTION  OF  XXX  FIGHTER  AIRCRAFT 

TAXI  XXX  FIGHTER  AIRCRAFT 

PERFORM  A  SHOREBASED  VFR  TAKE  OFF 

PERFORM  A  SHOREBASED  VFR  LANDING 

OPERATE  XXX  HEADS-UP  DISPLAY 

EXECUTE  FORMATION  MANEUVERS 

COMPLETE  POST-FLIGHT  REPORT 


Figure  4-3 . 


Task  Selection  (MS-3) . 


Select  Psychological  Variables 


ABORT 
No  More 
STIMULI 
RESPONSES 
FEEDBACK 


Figure  4-4.  Selection  of  Psychological  Variables  (MS-4) 


Psychological  variable:  Stimuli, 


Select  Attributes: 


-  ABORT 

-  All  Attributes 
MEDIUM 
VISUAL  FORM 
VISUAL  MOVEMENT 
VISUAL  SPECTRUM 
VISUAL  SCALE 
VISUAL  CONTRAST 
AUDIO  SOURCES 

AUDIO  STIMULI  INTENSITY 
STIMULI  PRESENTATION 
STIMULI  PRESENTATION  RATE 
NUMBER  OF  CHANNELS 
STIMULI  DISTRIBUTION 


Figure  4-5.  Stimulus  Variable  Selection  (MS-5) 


^v.v-v.v  v. w  .  .  •  • 


After  you  hit  Re  turn,  a  message  will  appear  on  the  screen 
asking  you  if  you  would  like  to  view  the  psychological 
variable  data.  Move  the  cursor  to  the  _No_  option  and  hit 

Return.  The  program  will  check  for  missing  data.  As  it 

does  so  the  name  of  each  task  in  the  example  will  appear  at 
the  top  of  the  screen.  (There  is  no  data  missing  in  this 
example. ) 

4.6  ACTION  6:  SELECTION  OF  PSYCHOLOGICAL  VARIABLES  (MS-4) 

You  will  return  to  the  frame  asking  you  to  select  another 

category  of  psychological  variables.  Move  the  cursor  to 

Responses  and  hit  Retur n. 

4.7  ACTION  7:  SELECTION  OF  RESPONSE  VARIABLES  (MS-6) 

This  frame  will  allow  you  to  select  response  variables. 
Move  the  cursor  to  the  All  Attributes  option  and  hit 
Return.  Then  move  the  cursor  to  No  More  and  hit  Return.  A 

message  will  then  appear  in  the  screen  asking  you  if  you 

would  like  to  view  the  response  variable  data.  Move  the 
cursor  to  _N°  an<3  hit  Return.  The  program  will  check  for 

missing  data.  As  it  does  so  the  name  of  each  task  in  the 

example  will  appear  at  the  top  of  the  screen.  (There  is  no 
data  missing  in  this  example.) 

4.8  ACTION  8:  SELECTION  OF  PSYCHOLOGICAL  VARIABLES  (MS-6) 

You  will  return  to  the  frame  asking  you  to  select  a  category 
of  psychological  variables.  Move  the  cursor  to  Feedback  and 
hit  Return. 


Select  Psychological  Variables 


ABORT 
No  More 
Stimuli 
Responses 
Feedback 


Figure  4-6.  Selection  of  Psychological  Variables  (MS-4) 


Psvcholoqical  variable:  Responses 


Select  attributes 


-  /JBORT 


-  All  Attributes 


RESPONSE  MODE 


INTENSITY  OF  RESPONSE 


RESPONSE  IMPLEMENTATION 


REQUIRED  RESPONSE  RATE 
#  RESPONSE  CHANNELS 


RESPONSE  DISTRIBUTION 


Figure  4-7.  Response  Variable  Selection  (MS-6) 


ABORT 


-  No  More 

-  Stimuli 

-  Responses 

-  Feedback 


Figure  4-8.  Selection  of  Psychological  Variables  (MS-4) 


4.9  ACTION  9:  SELECTION  OF  FEEDBACK  VARIABLES  (MS-7) 


This  frame  will  allow  you  to  select  Feedback  variables.  Move 
the  cursor  to  the  All  Attributes  option  and  hit  Return. 
Then  move  the  curosr  to  No  More  and  hit  Return.  A  message 
will  then  appear  on  the  screen  asking  you  if  you  would  like 
to  view  the  feedback  variable  data.  Move  the  cursor  to  No 
and  hit  Return.  (There  is  no  data  missing  in  this  example.) 


4.10  ACTION  10:  DATA  REVIEW  -  1  (MS-8) 

This  frame  is  designed  to  allow  you  to  deal  with  any  missing 
data  identified  during  the  previous  frame.  The  frame  will 
give  you  the  option  of  selecting  a  new  set  of  tasks  or 
exiting  to  enter  data  into  the  SDT. 

Since  you  have  no  data  missing,  move  the  cursor  to  Conti nue 
and  hit  Return. 


4.11  ACTION  11:  MEDIA  CATEGORY  SELECTION  (MS-13) 

This  frame  allows  you  to  select  the  media  categories  to  be 
used  in  the  analysis.  Move  the  cursor  to  Select  all  media 
categories  and  hit  Return.  Then  move  the  cursor  to  No  More 
and  hit  Return. 


4.12  ACTION  12:  COMPUTATION  OF  EFFICIENCY  SCORES  (MS-14) 

During  this  frame,  a  message  will  appear  on  the  screen, 
indicating  that  the  program  is  calculating  efficiency.  You 
do  not  have  to  do  anything  during  this  frame.  When  the 
scores  have  been  computed,  you  will  go  to  the  next  frame. 


Psychological  Variable;  Feedback. 


Select  Attributes 

-  ABORT 

-  ALL  ATTRIBUTES 
FEEDBACK  MEDIUM 
FEEDBACK  SOURCE 
FEEDBACK  TYPE 
FEEDBACK  DISTRIBUTION 


Figure  4-9-  Feedback  Variable  Selection  (MS-7) 
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You  have  just  reviewed  psychological  variables.  Please  select 
the  appropriate  option. 


-ABORT 
-Continue 
-Different  tasks 

-Exit  to  enter  missing  data  into  SDT. 


Figure  4-10.  Data  Review-1  (MS-9). 
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Please  select  built-in  media  categories  for  analysis 


-  ABORT 

-  All  Media  Categories 
Prnt  Ins  w/o  Fd 
Prnt  In  w/oral  Fd 
Prnt  In  w/oral  Fd 
Prnt  In  w/writ  Fd 
Prnt  -  Ref  Material 
Image  Proj  -  Still 
Passive  Audio 
Active  Audio 

AV  -  Still  w/Fd 
AV  -  Still  w/o  Fd 
AV  -  Motion  w/o  Fd 
AV  -  Mot  w/o  Fd 
Stat  Disp  w/o  Fd 
Dyn  Displ  w/o  Fd 
Phys  Train  -  inter 
Phys  Train  -  audio 
Phys  Train  -  visual 
-  more  - 


Figure  4-11.  Media  Category  Selection  (M-13) 


4.13  ACTION  13:  SELECTION  OPTION  FOR  VIEWING  EFFICIENCY/ 
EFFECTIVENESS/ "UTILITY"  SCORES  (MS-15) 

This  frame  provides  you  with  the  option  of  viewing 
efficiency  scores. 

Move  the  cursor  to  No  and  hit  Return. 


4.14  ACTION  14:  PRINT  CRITERIA  SCORES  (MS-19) 

This  frame  will  allow  you  to  print  out  the  efficiency 
scores.  Move  the  cursor  to  the  Ef f ic iency  option  and  hit 
Return.  The  efficiency  score  for  each  media  -  task 
combination  will  then  be  printed  on  your  printer. 


4.15  ACTION  15:  PRINT  CRITERIA  SCORES  (MS-19) 

Move  the  cursor  to  No  More  and  hit  Return.  A  message  will 
then  appear  on  the  screen  asking  you  if  the  results  are 
satisfactory.  Move  the  cursor  to  yes  and  hit  Return. 

At  this  point,  the  program  will  automatically  print  out  the 
task-media  assignments  which  maximize  efficiency.  In  so 
doing  it  will  assign  each  task  to  the  media  which  produces 
the  highest  efficiency  score. 


After  the  results  being  printed  the  Rerun  Option  (MS-23) 


if]'*] 


[»]•] 


Select  criteria  to  be  printed 


-Continue 

-Efficiency 


Figure  4-14.  Print  Criteria  (MS-19) 
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Select  rerun  option: 


-ABORT- 

-Select  different  criteria 
-Select  different  objectives 


Figure  4-15.  Select  Rerun  Option  (MS-23) 


SECTION  5  -  EVALUATING  AND  USING  RESULTS 


Av  overview  of  the  process  for  evaluating  and  using  the 
results  of  the  Media  Selection  Program  is  presented  in 

Figure  5-1.  There  are  four  steps  in  this  process.  In  the 

first  step,  sensitivity  analyses  are  conducted.  In  the 

second  step,  specific  types  of  media  within  each  media 
category  are  identified  for  each  task  -  media  category 
assignment.  In  the  third  step,  the  media  assignments  are 
reviewed  with  respect  to  a  set  of  practical  factors  which 
may  restrict  media  assignments.  In  the  fourth  step,  the 

final  set  of  media  assignments  are  documented  in  the  SDT. 

More  details  on  each  of  these  four  steps  are  provided  in  the 
sections  which  follow. 


5.1  STEP  1  CONDUCT  SENSITIVITY  ANALYSES 

As  the  first  step  in  analyzing  your  results,  you  may  want 
to  see  how  robust  your  analyses  are  to  changes  in  key 
analyses  alternatives  and  parameters.  The  more  consistent 
your  result  are  over  a  range  of  reasonable  alternatives  and 
parameter  values,  the  more  likely  it  is  that  your  results 
are  correct.  One  of  the  advantages  of  the  automated  Media 
Selection  Program  is  that  it  allows  you  to  quickly  determine 
the  impact  of  these  sensitivity  analyses.  Table  5-1 
displays  the  different  alternatives  that  may  be  explored  in 
a  sensitivity  analysis.  The  three  most  likely  alternatives 
you  will  want  to  explore  are:  (1)  alternative  criteria,  (2) 
alternative  objectives,  and  (3)  alternative  constraints. 


Figure  5-1  Guidelines  for  Evaluating  and  Using  Results 
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Table  5-1. 

Alternatives  for  Sensitivity  Analyses 

• 

CRITERIA 

• 

PSYCHOLOGICAL  VARIABLES 

• 

MEDIA  CATEGORIES 

• 

CRITICALITY  VARIABLES 

• 

OBJECTIVES 

• 

CONSTRAINTS 

• 

CRITICALITY  WEIGHTS 

• 

UTILITY  WEIGHTS 

In  planning  new  analyses,  make  sure  to  use  the  Media 
Selection  Program  planning  worksheet  listed  in  Table  2-3  to 
document  your  analyses. 

5.2  STEP  2:  IDENTIFY  SPECIFIC  MEDIA  WITHIN  CATEGORIES 

The  Media  Selection  Program  will  identify  the  generic 
category  of  media  which  should  be  used  to  train  a  task.  To 
develop  more  detailed  media  requirements,  specific  media 
within  each  of  these  categories  must  be  identified.  Table  5- 
2  displays  sane  examples  of  specific  media. 

Media  may  be  divided  into  two  general  groups  (1)  Course- 
Related  Media  -  those  that  are  developed  specifically  for  a 
particular  course  and  (2)  Course-Independent  Media  those 
that  are  supplied  by  the  local  school  and  used  in  wide  range 
of  courses.  Ifcble  5-3  displays  a  worksheet  which  can  be 

used  to  describe  the  specific  media  that  is  used  in  number 
of  different  courses.  Relevant  media  categories  within  each 
of  these  two  groupings  are  identified  in  Table  5-2.  To 
identify  specific  media  for  Course-Independent  Media,  you 
must  examine  the  specific  media  that  is  likely  to  be 

available  within  each  m-dia  category  at  the  school  where. the 
course  will  be  taught.  To  select  specific  media  for  Course- 
Related  Media,  you  must  identify  the  most  comparable 
existing  task  that  was  taught  via  the  same  media  category 

and  identify  the  specific  media  that  was  used  to  train  that 
task.  (Procedures  for  identifying  comparable  tasks  are 
described  in  the  ETES  User  Guide.)  Specific  media 

identified  for  the  Simulation- Three  Dimensional  Media 
Category  should  be  reviewed  with  PM  TRADE  since  this 
organization  is  likely  to  have  the  most  up-to-date  listing 
of  media  in  this  category. 


1.1  Print  Instruction  —  w/o  Feedback  Case  study  with  Folders,  Handouts,  Textbooks 

Visual  only  1.2  Print  Instruction  —  with  Oral  Feedback  Flash  Cards 
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Table  5.2.  Examples  of  Specific  Media  Categories. 
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5.3  STEP  3:  ASSESS  IMPACTS  OF  PRACTICAL  FACTORS 


Before  making  the  final  assignment  of  tasks  to  media,  you 
should  review  these  potential  assignments  against  practical 
factors  which  may  limit  or  in  some  cases  prohibit  making  the 
task-media  assignments  produced  by  the  Media  Selection 
Program.  More  specifically,  each  task-media  assignment  must 
be  reviewed  in  terms  of  the  practical  factors  listed  in 
Table  5-3.  After  reviewing  the  impacts  of  these  factors, 
the  final  assignments  of  tasks  to  media  can  be  made. 


5.4  ENTER  DATA  INTO  SDT 

Once  the  final  task-media  assignments  have  been  made,  these 
assignments  must  be  entered  into  the  SDT  data  base 
management  system.  To  do  this  you  must  enter  the  Input  mode 
of  the  SDT,  select  the  entity  Media  and  enter  the  titles  for 
all  of  the  specific  media  you  have  identified  in  your 
analysis.  While  still  in  the  Input  Mode  you  must  then 
select  the  entity  Course ,  select  the  subentity  Module  and 
the  attribute  media,  and  enter  the  information  required  to 
link  course  modules  to  media.  The  worksheet  listed  in  Table 
5-4  can  be  used  to  help  in  entering  media  selections  into 
the  SDT. 


Table  5-4. 
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1. 


Practical  Factors  to  Consider  in  Making 
Media  Assignments 


Marginal  Technical  Solutions  -  The  learning  guidelines 
and  algorithm  cannot  be  easily  carried  out  with  the 
system. 


2.  State-of-the-Art  -  The  system  is  under  development  at 
best  and  may  not  be  available  for  practical  application 
by  the  time  it  is  required. 


-■.*» 


3.  Size  of  the  System  -  Some  approaches  are  useful  within 
large  training  programs.  Others  are  suited  only  for 
small  programs  and  therefore,  may  not  be  suited  to  the 
size  program  being  considered. 


4*  Interface  with  Existing  Program  -  Many  new  courses  must 
be  designed  to  fit  into  existing  programs,  which  place 
constraints  on  the  new  courses;  e.g.,  equipment  on  hand, 
available  classrooms,  scheduling  practices. 

5.  Time  to  Produce  System  -  Approaches  which  require  long 
lead  times  for  development  may  not  be  useful  when 
scheduled  ready-for-training  dates  do  not  allow  a  long 
de ve 1 opmen  t  cycle. 

6.  Budget  Cycle  Constraints  -  While  the  application  of  some 
of  the  powerful  training  approaches,  such  as  CAI  (Computer 
Assisted  Instruction),  may  result  in  low  costs  per  student 
graduate,  the  initial  investment  is  substantial.  Unless 
these  resources  appear  in  existing  budgets,  the  applica¬ 
tions  of  these  techniques  to  an  immediate  problem  may  not 
be  feasible. 


V 


7.  Adoption  of  Innovations  -  Instructors  frequently  resist 
innovations.  If  the  proposed  technique  is  significantly 
different  from  existing  techniques,  either  adequate  re¬ 
sources  must  be  focused  upon  gaining  acc-^otance  for  the 
innovation,  or  a  more  traditional  approach  must  be  selected 

8.  Courseware  Development  -  If  the  courseware  is  to  be  locally 
developed,  skilled  personnel,  equipment,  time,  and  dollars 
must  be  made  available. 

9.  High  Cost  Alternatives  -  The  projected  life  cycle  cost  of 
a  media  approach  may  be  significantly  higher  than  other 
equally  useful  alternatives.  Reject  high  cost  alternatives 
when  others  are  available. 

10.  Learning  Style  of  Trainees  -  If  the  trainee  has  a  low 
reading  ability  or  would  be  limited  in  his  ability  to 
use  certain  kinds  of  systems  then  reject  these  systems 
as  inappropriate. 

11.  Other  Constraints  -  A  variety  of  other  practical  factors 
should  be  considered;  e.g.,  command  policy  and  existing 
investment  in  production  facilities. 


*  Training  Device  Requirements  and  Document  Guide. 
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APPENDIX  A  -  TECHNICAL  DESCRIPTION  OF  MEDIA 
SELECTION  PROGRAM  ALGORITHMS 


A.l  COMPUTING  THE  SCORES 

In  order  to  produce  an  optimal  assignment  of  media  to  tasks, 
some  measures  of  the  appropriateness  of  possible  assignments 
must  be  used  as  criteria.  The  four  basic  measures  used  by 
the  Media  Selection  Program  are  training  efficiency, 
training  effectiveness,  relative  cost,  and  "utility." 

A. 1.1  Training  Efficiency 

Training  efficiency  provides  a  measure  of  the  match  between 
the  characteristics  of  a  training  medium  and  the  character¬ 
istics  of  a  task  to  be  trained,  based  on  the  percentage  of 
the  psychological  variables  involved  in  the  task  which  are 
provided  by  the  training  medium.  This  approach,  taken  from 
the  TEEM  model  (Jorgenseon,  Kubula,  and  Atlas,  1981),  is 
detailed  below.  Given  T  tasks,  M  media,  and  P  psychological 
variables,  we  make  the  following  definitions: 

r  1  if  task  t  involves  psychological  variable  p 
Tt,p  =  i  0  otherwise 

.  r  1  if  medium  m  involves  psychological  variable  p 
Pm,p  =  ‘  0  otherwise 


The  [  t ]  matrix  is  an  input  from  the  SDT ;  the  [y]  matrix  is 

hard-coded  in  the  program.  From  these  matrices,  the 

efficiency  of  each  possible  task-medium  combination  (e,  ) is 

c  t  ,m 

computed  as  follows: 

1)  A  preliminary  score  <psfc  m)  is  computed  as 

P 

Pst,m  =  ^  Tt,p  Mm,p 

This  is  equal  to  the  number  of  psychological  variables  that 
task  t  and  medium  m  have  in  common. 

2)  An  ideal  score  (it)  is  computed  for  each  task  as 

P 


This  is  the  number  of  psychological  variables  involved  in 
task  t,  which  is  equal  to  the  best  score  possible  for  any 
medium  with  task  t. 

3)  Finally,  the  efficiency  is  computed  as 


where  "<  >"  denotes  the  greatest  integer  function  (the 

computed  number  is  rounded  down  to  the  nearest  integer). 
Thus  training  efficiency  is  an  integer  between  0  and  100 
(inclusive)  which  measures  the  appropriateness  of  a  given 
training  medium  for  training  a  given  task.  Training 
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efficiency  is  computed  for  each  possible  task-medium 
combination. 


A. 1.2  TRAINING  EFFECTIVENESS 

Training  effectiveness  (efc  )  ,  a  weighted  measure  of 
training  efficiency  based  on  the  criticality  of  the  task,  is 
computed  as  the  product  of  efficiency  and  task  criticality 
(kt).  Task  criticality  is  a  composite  of  eight  component 
criticality  variables:  frequency,  percent  performing, 

percent  time  performing,  task  delay  tolerance,  consequences 
of  inadequate  performance,  task  learning  difficulty,  time 
from  entry  to  performance,  and  probability  of  deficient 
performance.  The  overall  criticality  of  a  task,  kt  is  an 
integer  between  1  and  7  (inclusive)  where  1  represents 
minimum  criticality  and  7  represents  maximum  criticality 
(see  A. 1.2.1  for  a  detailed  explanation  of  how  the  kfc  are 
computed).  Effectiveness  is  computed  for  each  task-medium 
combination  as 

e . 
t  ,m 


Thus  training  effectiveness  is  an  integer  between  0  and  100 
( inclusive ) . 
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A. 1.2.1  Task  Criticality 

Overall  task  criticality  is  a  composite  of  eight  component 
criticality  variables  from  the  SDT  (see  Table  B-2).  Six  of 
the  eight  components  (frequency,  percent  time  performing, 
consequences  of  inadequate  performance,  task  learning 
difficulty,  time  from  entry  to  performance,  and  probability 
of  deficient  performance)  are  scaled  from  1  to  7,  with  a 
score  of  7  representing  maximum  criticality. 

One  of  the  components,  task  delay  tolerance,  is  scaled  from 
7  to  1,  with  a  score  of  1  representing  maximum  criticality. 
To  convert  this  score  from  a  7-to-l  scale  to  a  l-to-7  scale, 
the  converted  score  (c)  is  computed  from  the  unconverted 
score  (u)  as  follows: 


c  =  8  -  u. 


Another  of  the  component  criticality  variables,  percent 
performing,  is  scaled  from  0  to  100,  with  a  score  of  100 
representing  maximum  criticality.  To  convert  this  score  from 
a  0-to-100  scale  to  a  l-to-7  scale,  the  converted  score  (c) 
is  computed  from  the  unconverted  score  (u)  as  follows: 


7)  +  99 


Once  the  component  criticality  scores  are  on  the  same  scale, 
a  composite  criticality  may  be  computed  for  each  task  as  a 
weighted  average  of  the  eight  components.  Eight  criticality 
variable  weights  (w^)  are  entered  by  the  user;  each  weight 
is  between  0  and  100  (inclusive),  and  the  sum  of  the  eight 


weights  must  be  100.  Then  the  criticality  of  task  t  (kt)  is 
computed  as 


where  <.  .  is  the  criticality  score  of  task  t  on  the  ith 
X  f  z. 

component  of  criticality,  and  w^  is  the  user-specified 
weight  for  the  ith  criticality  component. 

✓ 

A. 1.3  Relative  Cost 

Each  of  the  media  in  the  Media  Selection  Program  is  assigned 
a  relative  cost  (cm).  The  cm  are  between  0  and  100 
(inclusive)  where  100  represents  maximum  cost.  Clearly, 
these  costs  are  not  in  actual  dollars?  but  they  are  based  on 
the  dollar  costs  of  the  media. 

A. 1.4  "Utility" 

In  considering  possible  task-medium  assignments,  it  is 
useful  to  combine  efficiency  or  effectiveness  with  cost  into 
one  measure  of  the  overall  goodness  (or  "utility")  of  task- 
medium  combinations.  The  user  specifies  weights  (wc  and  we) 
which  reflect  the  relative  importance  of  cost  and 
efficiency/effectiveness;  the  weights  must  be  between  0  and 
100  (inclusive)  and  must  sum  to  100.  Then  "utility"  is 
computed  for  each  task-medium  combination  as  follows: 


* 


[w. 


(100  -  cm)  ]  +  [wQ  * 


t  f  in 


t  ,m 


or 


100 


(100  -  c  )]  +  [w  * 
m  e 


100 
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depending  on  whether  "utility"  (ufc  ^ )  is 

efficiency  (efc  ffl)  or  "utility"  (ut,m)  is 

effectiveness  (e.  ). 

t ,  m 


to 

be 

based  on 

to 

be 

based  on 

A. 2  OPTIMIZATION 

Two  forms  of  optimization  are  available  to  the  user  of  the 
Media  Selection  Program:  unconstrained  and  constrained. 


A. 2.1  Unconstrained  optimization 

An  unconstrained  optimization  will  occur  whenever  the  user 
selects  an  objective  and  "no  constraint."  For  example,  if 
the  user  selects  "maximize  efficiency"  as  his  objective,  and 
"no  constraint,"  the  program  will  return  the  unconstrained 
maximum  efficiency  assignment.  Mathematically  this  problem 
is  formulated  and  solved  as  follows. 

Given  T  tasks  and  M  media,  and  an  efficiency  et,m  for  each 
task-medium  combination,  each  task  will  have  assigned  to  it 
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the  medium  with  the  greatest  efficiency  for  that  task 
according  to  the  following  rule: 


I 

Assign  medium  m  to  task  t  only  if: 


©•-  m  >  e*.  <  for  all  i£  {1,2, 
t,m  t,i 


The  rules  for  the  other  optimizations  are  similar  to  that 
for  efficiency:  To  maximize  effectiveness,  assign  medium  m 
to  task  t  only  if 


«  >  ©,.  i  for  all  ie  {1,2, 

c  t  m  u  t  x 


To  maximize  "utility"  assign  medium  m  to  task  t  only  if 


u*.  „  >  u.  .  for  all  ie  {1,2,...M}. 
t  ,m  t ,  l 


To  minimize  relative  cost,  assign  medium  m  to  all  tasks  only 
if 


cm  <  ci  £or  a11 


Note  that  relative  cost  is  task-independent. 
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A. 2. 2  Constrained  Optimization 


A  constrained  optimization  will  occur  whenever  the  user 
selects  both  an  objective  and  a  constraint.  The 

optimization  is  performed  using  a  dynamic  programming 
algorithm  which  specifies  the  optimal  assignment  of  exactly 
one  medium  to  each  task  for  each  of  a  discretized  set  of 
levels  of  the  constraint.  Detailed  discussions  of  the 
formulation,  the  dynamic  programming  algorithm,  and  the 
output  follow. 


A. 2. 2.1  The  Formulation 


Assume  for 
efficiency" 
constraint, 
as  follows: 


example  that  the  user  selects  "maximize 
as  the  objective,  and  "cost"  as  the 
The  mathematical  formulation  for  this  option  is 


max 


T 

Z  e.  m 
t=l  t'rat 


T 

st  Z 
t  =  l 


'm. 


<  c. 


where  mt  is  the  medium  assigned  to 
give  the  value  of  the  objective 
as  fT(cQ))  and  a  list  of  the  optima 
(ic  {1,2, ...,M})  for  each  task  t  = 


task  t.  A  solution  must 
function  (later  defined 
1  assignment  dt  = 

1,2, ...,T. 
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A.2.2.2  The  Dynamic  Programming  Solution  Approach 


For  T  tasks  and  M  media,  there  are  efficiencies  (et,m)  for 
each  task-medium  combination,  and  costs  (cm)  for  each 
medium.  The  optimization  must  assign  exactly  one  of  the  M 
media  to  each  of  the  T  tasks.  For  task  1  and  a  level  of 
cost,  z,  define  the  set  Dj^(z)  as 


D1(z)  s  {m|cm<z}. 


Then  D^tz)  is  the  set  of  media  with  costs  not  greater  than 
z,  or  the  set  of  feasible  media  for  task  1,  cost  z.  Define 
f i ( z )  as 


f  2  (  z  )  =  maximum  {e-^ /inJ  meDj  (  z  )  }  . 


Then  fx(z)  is  the  greatest  attainable  efficiency  for  an 
assignment  of  task  1  to  a  medium  with  cost  not  greater  than 
z . 

Next,  define  the  set  D2(z)  as 


D2(z)  =  {m|D1(z  -  cm)  *  0}. 


where  0  indicates  the  null  set 


Then  D2(z)  is  the  set  of  feasible  media  for  task  2,  cost 
z.  Note  that  D2  is  dependent  only  on  z,  D^,  and  the  cm. 
Next,  define  f2(z)  as 


f2(z)  =  maximum  (S2,m  +  ^l^z  ~  cm)  |  msDjC  z  )  } 


Then  f2(z)  is  the  greatest  attainable  efficiency  for  an 
assignment  of  tasks  1  and  2  to  media  with  a  total  cost  not 
greater  than  z. 

Then,  in  general,  Dt(z)  is  defined  as 


Dt(z)  =  {m|Dt_1(z  -  cm)  *0 } , 


and  ft(z)  is  defined  as 


ft(z)  =  maximum  +  ft-l^z  “  cm) ! meDt( z ) } , 


Again,  note  that  Dt  is  dependent  only  on  z,  and  the 


This  function,  f,  is  a  dynamic  programming  recursion  in 
canonical  form.  In  particular,  fT(c0)  gives  the  maximum 
attainable  efficiency  for  an  assignment  of  all  T  tasks  to 
training  media  with  the  total  cost  of  the  assignment  not 
greater  than  c0.  The  optimal  assignment  of  media  to  tasks 


is  then  obtained  by  back-substitution  ("unwinding  )  in  the 
above  equation  (see  A. 2. 2. 3). 

The  above  example  gives  the  formulation  for  efficiency  as  an 
objective,  and  cost  as  a  constraint.  In  general,  given  T 
tasks,  M  media,  a  matrix  of  objective  values  for  each  task- 
medium  pair  (otfin),and  a  matrix  of  constraint  values  (st>m) 
for  each  task-medium  pair,  the  dynamic  programming  recursion 
may  be  formulated  as  follows: 


Let  (z)  = 


{mlsi,m  *  z)  if  the  constraint  is  cost, 
{m|Si  m  >  z }  otherwise. 


and  for  te { 2 , 3 , . . . ,T} ,  define  Dt(z)  as 


Dt  <*>  h  tm I ^t-l ^ 2  ~  st,m>  *0 


Then  fj(z)  is  defined  as 


f^(z)  =  optimum  {o^m  |  m  eD^  (  z  )  } , 


and  for  te  {2,3,. ..,T),  f(-(z)  is 


ft(z)  =  optimum  {ot,m  +  f  t— 1  (2  ~  sm,  t  2  ^ ' 


where  "optimum"  means  maximum  or  minimum  depending  on  the 
objective  selected,  minimum  for  cost  and  maximum  otherwise. 

A.2.2.2.  The  Algorithm 

As  stated  earlier,  the  Media  Selection  Program  determines 
the  optimal  assignment  for  a  discretized  set  of  50  levels  of 
the  constraint  within  the  range  of  feasible  constraints. 
This  is  accomplished  by  execution  of  the  following  steps: 

Step  1:  Determine  the  minimum  constraint  level  (sm^n)  for 

which  an  assignment  of  any  task  to  any  medium  is  possible  as 


Ste2_4:  For  zs{smin>  smin  +  a,  smin  +  20 . smin  +  50a}, 

make  a  table  of  triples  (z,f^(z),  a^(z))  where  a^(z)  =  m 
only  if  f^(z)  =  °i,m*  Then  a^(z)  is  the  optimal  medium  for 
task  1  with  a  constraint  level  of  z. 


Step  5:  Perform  the  following  for  t=2,3,...,T: 


For  Z£(smin'  smln'  +  »' 


smin  +  2°'- 


table  of  triples  (z,  ft(z),  at(z)) 
et<z>  "  °t ,m* 


. sm^n  +  50a}/  /  make  a 
Where  at(z)  =  m  only  if 


When  step  5  is  complete,  the  table  for  t=T  contains  a  triple 
for  each  of  50  levels  of  the  constraint.  Each  triple  gives 
a  constraint  level  (z),  the  optimum  objective  value 
attainable  under  the  constraint  (ft(z)),  and  the  optimal 
assignment  for  task  T  under  the  constraint  (at(z)). 


A. 2. 2. 3  The  Output 


Once  the  optimization  has  been  performed,  the  Media 
Selection  Program  prints  out  a  part  of  the  table  generated 
in  step  5  above  for  T=t.  Table  A-l  is  an  example  of  the 
printout.  The  printout  shows  the  user  how  much  objective 
value  he  can  "get,"  given  the  associated  constraint  level, 
i.e.  ft(z)  vs.  z.  Associated  with  each  constraint  level  is 
an  assignment  of  all  tasks  to  media.  When  the  program 
queries  the  user  for  the  constraint  level  for  which  he  would 
like  to  see  the  assignment,  the  user  may  enter  a  number 
between  smin  and  smax.  The  program  then  "unwinds"  the 
tables  to  produce  the  optimal  assignment  for  the  user- 
specified  constraint  level  as  follows: 


we 
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Table  A-l.  Example  of  Printout  of 
Constraint  and  Objective  Variable  Scores 

OBJECTIVE:  MAXIMIZE  EFFICIENCY 

CONSTRAINT:  MINIMIZE  COST 


CONSTRAINT  LEVEL 


OBJECTIVE  VALUE 


Let  cD  be  the  constraint  level  entered  by  the  user.  Then 
task  T  is  assigned  to  media  at(cQ).  Define  rT  to  be  the 
remaining  amount  of  constraint  after  task  T  is  assigned,  or 


rT  5  Co  ~  ST,  aT  <cQ) 


Then  task  T-l  is  assigned  to  media  aT_^(rt).  And 


rT-l  H  rT  "  ST-1,  aT-1  (rT) 


is  the  remaining  amount  of  constraint  after  tasks  T  and  T-l 
have  been  assigned.  Similarly,  rfc  for  te  (1 , 2 , . . . ,T-1  }  is 
defined  as 


rt  3  rt+i 


'  St'at  "W 


Then,  in  general,  the  medium  to  be  assigned  to  each  of  the 
tasks  t  £  {T-l  ,T-2 ,  .  .  .  ,  1  }  is  at^rt  +  l^  =  ^t*  These  are  the 
assignments  that  are  printed  out  in  Table  A-2. 


A. 3  TECHNICAL  NOTES 

A. 3.1  The  "Utility"  Function 

The  function  used  to  combine  cost  and  efficiency/ 
effectiveness  (see  A. 1.3)  cannot  legitimately  be  called  a 
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utility  function  in  the  vonNeumann-Morgenstern  sense;  no 
method  is  provided  to  elicit  from  the  user  the  data 
necessary  to  construct  a  function  which  satisfies  the  axioms 
of  utility  theory.  Rather,  the  user  is  provided  with  the 
most  straight-forward  preference  curve:  the  straight  line  of 
expected  value. 


A. 3. 2  Accuracy  of  the  Dynamic  Programming  Solution 

In  order  to  keep  main  memory  requirements  within  Apple  III 
limits,  all  possible  constraint  levels  are  not  considered. 
Instead,  the  feasible  region  is  represented  by  50  equally- 
spaced  constraint  levels  (see  A.2.2.2).  This  means  that  a 
worst  case  error  of  up  to  <j*T  may  be  introduced  into  the 
constraint  levels  reported  for  the  possible  assignments  (see 
Table  A-l ) ,  the  average  error  will  be  much  smaller. 
However,  this  is  not  a  severe  problem,  as  the  exact 
constraint  level  is  recomputed  and  printed  for  any  "unwound" 
assignment  (see  A. 2. 2. 3). 


A. 3. 3  Integers  vs.  Real  Numbers  n  the  Apple  III 

An  Apple  Pascal  library  unit  is  available  to  process  real 
numbers,  but  it  requires  a  great  deal  of  main  memory  to 
run.  Since  the  memory  requirements  of  the  Media  Selection 
Program  are  quite  large,  the  Apple  library  unit  for  real 
numbers  cannot  be  used  and  all  computations  are  done  using 
integer  arithmetic.  Therefore  all  values  computed  using  a 
division  are  subject  to  a  relatively  small  truncation  error. 


This  appendix  provides  definitions  for  the  psychological  and 
criticality  variables  used  in  the  Media  Selection  Program. 
In  addition,  it  lists  the  relative  cost  values  for  each 
media  category  used  in  the  program. 

Table  B— 1  provides  definitions  for  the  psychological 
variables.  Those  definitions  are  taken  directly  from 

Jorgensen,  Kubulas,  and  Athus  (1981).  Most  of  these 
definitions  were  derived  from  the  TRAD  PC  Task  Analysis 
Handbook.  Table  B-2  provides  definitions  for  the 

criticality  variables.  Table  B-3  lists  the  relative  cost 
values  for  each  media  category  used  in  the  program. 


Table  B-l.  Psychological  Variables 


* 


STIMULUS  VARIABLES 


Medium  of  Stimuli  Presentation 


1.  Visual  Cues  -  Signals  received  through  the  sense  of  sight. 

2.  Audio  Cues  -  Signals  received  through  the  sense  of  hearing. 

3.  Tactile  Cues  -  Signals  received  through  the  sense  of  touch,  including 
sensations  related  to  texture,  size,  shape,  or  vibration  of  the  skin. 

4.  External  Stimulus  Motion  Cues  -  The  sensations  felt  by  a  person  when 
he  is  moved  by  some  outside  force  in  such  a  way  that  his  body  experiences 
roll,  pitch,  yaw,  heave,  sv/ay  and/or  surge. 

5.  Internal  Stimulus  Motion  Cues  -  The  sensations  felt  by  a  person  when 
he  moves  . lis  arm,  leg,  fingers,  etc. 

6.  Olfactile  Cues  -  Signals  received  through  the  sense  of  smell. 

7.  Gustatile  Cues  -  Signals  received  through  the  sense  of  taste. 

Visual  Form 

8.  Visual  Alphanumeric  -  Words  and/or  numbers  presented  visually. 

9.  Visual  -Symbolic  -  Symbols  presented  graphically. 

10.  Visual  Graphic  -  Two-dimensional  figures,  such  as  maps,  graphs, 
mathematical  curves,  etc.,  presented  visually. 

11.  Visual  Pictorial  -  Two-dimensional  images,  such  as  photographs, 
drawings,  etc.,  presented  visually. 

12.  Visual  Solid  Object  -  A  three-dimensional  image  or  reality  that  is 
viewed  from  exterior  perspectives. 

13.  Visual  Environment  -  A  three-dimensional  image  or  reality  that  is 
v i ewed  from  inside. 

Visual  Movement 


14.  Visual  Still  -  A  static  visual  field,  as  with  a  still  photograph, 
drawing  or  printed  page. 

15.  Visual  Limited  Movement  -  A  basically  static  visual  field  with  elements 
that  can  be  made  to  move,  as  with  an  animated  transparency  or  simple  panel 
with  switches  that  move. 
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Table  B-l.  (continued) 


16.  Visual  Full  Movement  -  A  visual  field  in  which  all  elements  can  move, 
as  with  a  motion  picture,  flight  simulator,  or  operational  aircraft. 

17.  Visual  Cyclic  Movement  -  A  visual  field  which  moves  through  a  fixed 
sequence  and  then  repeats  the  sequence  in  a  repetitive  manner,  as  with  a 
film  loop. 


Visual  Spectrum 


18.  Black  and  White  -  A  visual  field  composed  of  either  black  or  white 
elements,  as  with  the  printed  page  or  line  drawings. 


19.  Gray  Scale  -  A  visual  field  composed  of  black,  white  and  continuous 
gradations  of  gray,  as  with  a  black  and  white  photograph  or  television 
picture. 


20.  Color  -  A  visual  vield  composed  of  various  segments  of  the  visual 
spectrum,  as  with  color  television  or  motion  pictures. 


Visual  Scale 

21.  Exact  Scale  -  Actual  visual  field  or  a  one-to-one  replication  of 
that  field  as  with  a  full-sized  mock-up,  simulator,  or  operational  system. 

22.  Proportional  Scale  -  A  representation  of  reality  in  other  than 
full  scale,  such  as  a  scaled  model  map  or  photograph. 


Visual  Contrast 

23.  Dim  -  A  visual  object  which  blends  in  with  its  background,  i.e., 
there  is  a  small  or  no  luminance  difference  between  an  object  and  its 
background. 

24.  Bright  -  A  visual  object  which  is  brighter  than  its  background, 
i.e.,  there  is  a  large  luminance  difference  between  an  object  and 
its  background. 


Audio  Sources 

•• 

25.  Tonal  Sound  -  A  very  limited  source  of  sound  or  noise  which  is  used, 
rather  than  speech,  for  signaling  or  warning,  e.g.,  horns,  whistles,  sirens, 
bells,  buzzers,  etc. 

26.  Voice  Sound  -  A  limited  source  of  sound  which  enables  spoken  words 
to  be  used  as  the  medium  of  communications,  but  not  suited  to  more 
demanding  tasks,  such  as  music  or  sound  recognition  exercises. 

27.  Full  Sound  -  A  source  of  sound  that  contains  all  che  significant 
elements  of  the  sound  and  is  suited  to  the  demandinq  task  of  sound 
recognition  exercises. 


Table  B-l.  (continued) 


28.  Ambient  Sound  -  A  complex  sound  environment  with  sounds  emanating 
from  various  sources  and  from  various  directions,  including  background 
noise  and  task  significant  sounds. 

Audio  Stimuli  Intensity 

29.  Weak  -  Audio  stimuli  presented  to  the  trainee  which  have  weak  intensity. 

30.  Strong  -  Audio  stimuli  presented  to  the  trainee  which  have  strong 
intensity. 

Stimuli  Presentation 

.  .  .  —  —  -  _  -  -  ^ 

31.  Static  -  A  unitary  stimuli  situation,  i.e.,  stimuli  are  presented  to 
the  trainee  "all  at  once",  e.g.,  batch  presentations. 

32.  Dynamic-Ordered  -  A  sequential  stimuli  situation,  i.e.,  stimuli  are 
presented  to  the  trainee  sequentially  or  in  an  ordered  manner  over  time. 

33.  Dynamic- Random  -  A  non-sequential  stimuli  situation,  i.e.,  stimuli 
are  presented  to  the  trainee  randomly  over  time. 

Stimuli  Presentation  Rate 

34.  Slow  Rate  -  A  slow  rate  or  speed  of  presentation  of  stimuli  to  the 
trainee,  allowing  the  trainee  a  long  or  maximum  stimulus  analysis  time. 

35.  Fast  Rate  -  A  fast  rate  or  speed  of  presentation  of  stimuli  to  the 
trainee,  allowing  the  trainee  a  short  or  minimum  stimulus  analysis  time. 

Humber  of  Channels  or  Sources 

36.  Limited  -  A  small  number  of  sources,  channels,  or  instruments 
through  which  stimuli  are  presented  to  the  trainee. 

37.  Unlimited  -  A  multiple  number  of  sources,  channels,  or  instruments 
through  which  stimuli  are  presented  to  the  trainee. 

Stimuli  Distribution 

38.  Individual  -  All  information  is  presented  directly  to  one  individual 
trainee. 

39.  Group  -  Information  is  presented  to  a  group  of  trainees,  allowing  cily 
indirect  access  to  the  information  for  an  individual. 
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Table  B-l.  (continued) 


RESPONSE  VARIABLES 


Response  Mode  of  Implementation 

1.  Overt  Response  -  Verbal  -  A  response  which  the  trainee  expresses  in  an 
audible  (verbal)  manner,  such  as  a  verbal  short  answer  response  to  a  question 
having  a  limited  set  of  correct  answers,  a  conversational  response,  or  a 
verbal  decision  response. 

2.  Overt  Response  -  Written  -  A  response  which  the  trainee  expresses 

in  an  observable  (written)  manner,  such  as  a  free  style  written  response, 
a  written  multiple  choice  response,  or  a  written  fill -in~the  blank  response. 

..  Overt  Response  -  Manipulative  Acts'"-  A  response  which  the  trainee  expresses 
in  an  observable  (manipulative)  manner,  such  as  the  small  movements  of  dials, 
switches,  keys,  or  small  adjustments  to  instruments  or  the  large  movements  of 
levers,  wheels  or  use  of  hand  held  tools. 

4.  Overt  Response  -Tracking  -  A  response  which  the  trainee  expresses  in  an 
observable  (tracking)  manner,  such  as  continuously  controlling  a  constantly 
changing  system,  e.g.,  steering  an  automobile. 

5.  Overt  Response  -  Procedural  Performance  -  A  response  which  the  trainee 
expresses  in  an  observable  (procedural  performance)  manner,  such  as 
performing  a  sequence  of  steps  in  a  procedure,  e.g.,  carrying  out  the  items 
on  the  checklist  for  preflighting  an  aircraft  or  turning  on  a  radar  system. 

Intensity  of  Response 

•  Weak  -  Responses  made  by  the  trainee  with  weak  intensity. 

7.  Strong  -  Responses  made  by  the  trainee  with  strong  intensity. 

Response  Implementation 

8.  Static  -  A  unitary  response  situation,  i.e.,  responses  are  made  by  the 
trainee  "all  at  once". 

9.  Dynamic-Ordered  -  A  sequential  response  situation,  i.e.,  responses  are 
made  by  the  trainee  sequentially  or  in  an  ordered  manner  over  time. 

10.  Dynamic- Random  -  A  non-sequential  response  situation,  i.e.,  responses 
are  made  by  the  trainee  randomly  over  time. 
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Table  B-l.  (continued) 


Required  Response  Rate 

11.  Slow  Rate  -  A  slow  rate  or  speed  of  trainee  response,  i.e.,  a  rate  ’which 
allows  the  trainee  a  long  or  maximum  response  time. 

12.  Fast  Rate  -  A  fast  rate  or  speed  of  trainee  response,  i.e.,  a  rate  which 
allows  the  trainee  a  short  or  minimum  response  time. 

Humber  of  Response  Channels 

13.  Limited  -  A  limited  number  of  sources,  channels,  or  instruments  through 
which  required  responses  are  made  by  the  trainee. 

14.  Unlimited  -  An  unlimited  number  of  sources,  channels,  or  instruments 
through  which  responses  are  made  by  the  trainee. 

Response  Distribution 

15.  Individual  -  One  individual  trainee  makes  the  required  response. 


16.  Group  -  A  group  of  trainees  make  the  required  response. 


Table  B-l.  (continued) 


FEEDBACK  VARIABLES 


Medi um  of  Feedback  Presentation 


1.  Visual  -  Feedback  presented  visually  by  means  of  a  display,  it  may  be 
coded  and  transmitted  visually  to  the  trainee. 

2.  Aural  -  Feedback  presented  aurally  by  means  of  a  display  to  the  trainee. 

3.  Written  Form  -  Feedback  presented  tc  the  trainee  in  written  form. 

4.  Face-to-Face  Communication  -  Feedback  presented  by  direct  verbal  means 
to  the  trainee. 

j.  Indirect  Communication  -  Feedback  presented  by  indirect  verbal  means, 
such  as  by  intercom,  telephone,  or  radio  link. 

6.  Tactile  -  Feedback  presented  to  the  trainee  through  the-  sense  of  touch, 
including  sensations  related  to  texture,  shape,  size,  or  vibration  of  the  skin 

7.  Ki nestheti c  -  Feedback  presented  to  the  trainee  by  either  internal  or 
external  bodily  movement,  such  as  reaching,  grasping,  tilting,  etc. 

8.  01  tactile  -  Feedback  presented  to  the  trainee  through  the  sense  of  smell. 

9.  Gustati le  -  Feedback  presented  to  the  trainee  through  the  sense  of  taste. 
Source  of  Feedback 

10.  Intrinsic  F  -  Information  or  cues  built  into  the  system  from  which  the 
.rainee  interprets  feedback  information. 

11.  Extrinsic  F  -  Information  or  cues  not  inherent  in  the  trainee  action  or 
system  operations  but  is  supplied  by  an  external  source. 

Type  of  Feedback 

12.  Research  Correctness  (Rcr)  -  Information  about  the  correctness  or 
incorrectness  of  trainee's  response,  when  several  response  alternatives  are 
possible  and  the  correct  choice  is  not  known  to  the  trainee  in  advance. 

(Also  known  as  augmented  feedback.; 

13.  Response  Correctness  (Ref)  -  Information  provided  to  the  trainee  (or 
Dthers  who  need  to  know  about  his  performance)  that  he  has  in  fact  performed 
in  operation,  but  does  not  say  anything  about  the  longer  range  consequences 
?f  the  action  taken. 
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Table  B-l.  (continued) 


14.  Response  Consequences  (Ren)  -  Information  about  the  consequences  of  the 
action  taken.  It  confirms  the  response  made  by  the  trainee,  and  the 
correctness  of  a  response  can  be  inferred  only  from  its  consequences.  May 
also  serve  to  cue  the  trainee  to  perform  the  next  response  in  sequence. 

15.  System  Status  fRssl  -  Information  about  "he  condition  of  one's  own 
or  another  system  or  the  external  environment,  on  the  basis  of  which  a 
trainee  or  team  must  act.  Information  is  not  necessarily  (or  even  frequently) 
the  immediate  consequence  of  or  follow-on  to  a  specific  trainee/team  action; 
it  may  reflect  system  events  that  have  been  put  in  motion  by  much  earlier 
trainee  actions.  Provides  information  that  regulates  trainee  and  system 
actions  in  the  sense  that  when  a  particular  status  condition  occurs,  the 
trainee  must  often  take  action  to' maintain  the  integrity  of  his  system. 

Feedback  Distribution 


16.  Individual  -  Feedback  is  presented  to  one  individual  trainee. 

17.  Group  -  Feedback  is  presented  to  a  group  of  trainees,  allowing  only 
indirect  access  for  an  individual. 


Table  B-2.  Criticality  Variables1 


1.  Frequency  -  The  number  of  times  the  task  is  performed  per 

week . 

2.  Percent  Performing  -  The  percentage  of  soldiers  within  an 

MOS  who  are  performing  the  task. 

3.  Percent  Time  Performing  -  An  estimate  of  percentage  of  the 

average  workweek  that  soldiers  in  a  given  MOS  spend  in 
performing  the  task. 

4.  Task  Delay  Tolerance  -  A  measure  of  how  much  delay  can  be 

tolerated  between  the  time  the  need  for  task  performance 
becomes  evident  and  the  time  actual  performance  must 
begin.  (TRADOC  MPA  350-30) 

5.  Consequences  of  Inadequate  Performance  -  The  possible  adverse 

effects,  in  terms  of  personal  safety,  mission  success 
and/or  damage  to  the  system,  which  could  result  from 
inadequate  performance  of  a  task. 

6.  Task  Learning  Difficulty  -  An  estimate  of  the  time,  effort, 

and  assistance  required  by  a  student  to  achieve 
performance  proficiency. 

7.  Time  from  Entry  to  Performance  -  Amount  of  time  between  job 

entry  and  task  performance. 

8.  Probability  of  Deficient  Performance  -  An  estimate  of  the 

frequency  that  a  task  is  performed  poorly. 


1-More  details  on  obtaining  values  for  these  variables  are  listed 
in  the  ETES  User's  Guide. 


Table  B-3.  Relative  Cost  Values. 


CAT 

* 

GROUP 

# 

MEDIA  CATEGORY 

RELATIVE 

COST 

1 

1.1 

PRINT  INSTRUCTIONS  W/O  FEEDBACK 

1 

2 

1.2 

PRINT  INSTRUCTIONS  WITH  ORAL  FEEDBACK 

1 

3 

1.3 

PRINT  INSTRUCTION  -  WITH  WRITTEN  FEEDBACK 

1 

4 

1.4 

PRINT  -  REFERENCE  MATERIAL 

5 

5 

1.5 

IMAGE  PROJECTION  -  STILL 

1 

6 

2.1 

PASSIVE  AUOIO 

1 

7 

2.2 

ACTIVE  AUDIO 

4 

3 

3.1 

AUDIOVISUAL  -  STILL  WITH  FEEDBACK 

3 

9 

3.2 

AUDIOVISUAL  -  STILL  WITHOUT  FEEDBACK 

2 

ia 

3.3 

AUDIOVISUAL  -  MOTION  WITH  FEEDBACK 

5 

11 

3.4 

AUDIOVISUAL  -  MOTION  WITHOUT  FEEDBACK 

.  4 

12 

4.1 

STATIC  DISPLAY  WITHOUT  FEEDBACK 

1 

13 

A2 

DYNAMIC  DISPLAY  WITHOUT  FEEDBACK 

5 

14 

4.3 

PHYSIOLOGICAL  TRAINER  -  INTERNAL 

50 

15 

4.4 

PHYSIOLOGICAL  TRAINER  -  AUDIO 

37 

16 

4.5 

PHYSIOLOGICAL  TRAINER  -  VISUAL 

37 

17 

4.8 

SYMBOLIC  SIMULATION  WITH  FEEDBACK 

5 

18 

4.7 

TRAINER 

55 

19 

4.8 

SIMULATOR 

72 

20 

4.9 

OPERATIONAL  EQUIPMENT 

82 

21 

4.10 

OPERATIONAL  EQUIPMENT  WITH  FEEDBACK 

95 

22 

5.1 

VISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 

12 

23 

5.2 

VISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 

8 

24 

5.3 

VISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 

24 

25 

5.4 

VISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 

14 

26 

5.5 

AUDIOVISUAL  COMPUTER  -  STILL  WITH  FEEDBACK 

22 

27 

5.6 

AUDIOVISUAL  COMPUTER  -  STILL  WITHOUT  FEEDBACK 

18 

28 

5.7 

AUDIOVISUAL  COMPUTER  -  MOTION  WITH  FEEDBACK 

34 

29 

5.8 

AUDIOVISUAL  COMPUTER  -  MOTION  WITHOUT  FEEDBACK 

24 

30 

6.1 

INSTRUCTOR 
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